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ABSTRACT 



The goals of this assessment of electronic government 
information products were to:’’ident medium, and format standards that are 
the most appropriate for permanent public access;' assess the- 
cost-effectiveness and usefulness of various alternative medium/format 
standards; and identify public and private medium/format standards that could 
be used for products throughout their entire information life cycle. The 
study utilized both quantitative and qualitative data collection activities: 
a survey of a cross-section of 314 government information products from 24 
agencies, site visits to three federal depository libraries, meetings with 
four federal agencies, and interviews with six experts. Findings were 
analyzed in the following areas: (1) preferred mediums and format standards; 

(2) evaluating World Wide Web sites; (3) cost-effectiveness of formats and 
mediums; (4) depository library needs; and (5) public access, including 
permanent public access to and permanent retention of electronic government 
information, perspectives on permanent public access and information life 
cycle management from information resources management experts, and current 
initiatives on permanent public access and permanent retention. Appendices 
include lists of participating agencies, products surveyed, and experts, as 
well as the product assessment questionnaire and site visits data, an agency 
meeting agenda, and expert interview questions. (Contains 53 references.) 
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March 30, 1999 



Mr. Michael F. DiMario 
The Public Printer 
The Government Printing Office 
North Capitol and H Sts. NW 
Washington, D.C. 20401 

Dear Mr. DiMario, 

It is with great pleasure that I forward herewith a copy of the Final Report prepared by Westat, Inc., 
the contractor selected by the Government to undertake Phase II of the three-part study called “Assessment 
of Electronic Government Information Products.” As you requested, the U.S. National Commission on 
Libraries and Information Science (NCLIS) planned and implemented this research survey, pursuant to an 
interagency agreement between NCLIS and the Government Printing Office (GPO), approved by the Joint 
Committee on Printing (JCP). 

This report follows on the process begun with the congressional requirement, contained in the Senate 
Report on H.R. 1854, the FY 1996 Legislative Branch Appropriations Act (P.L. 104-53), to identify the 
measures necessary for a successful transition to a more electronic Federal Depository Library Program. 
That requirement resulted in a study published by the Government Printing Office in June 1996. There was 
a consensus, however, that additional work was required (1) to identify the electronic formats and mediums 
used and/or planned by Federal publishing entities, and (2) to determine whether public or private sector 
standards do, or could, play a stronger role in reducing the unnecessary proliferation of these formats and 
mediums. These questions precipitated this survey. 

I am extremely pleased to note that the survey enjoyed the active support and participation of all 
three branches of Government. Twenty-four different Federal entities participated, including the Supreme 
Court, several committees of the Congress, one regulatory commission and 19 Executive Branch agencies, 
including most of the Cabinet Departments. In addition to this broad and diverse Federal involvement in 
the survey, an impressive 74 percent of the survey forms sent to the agencies were returned completed. I 
believe this level of interest and support is highly unusual, and could, perhaps, be construed as a reflection 
of agency desires to help establish a systematic baseline for measuring and monitoring the rapidly changing 
and evolving kinds and mix of preferred mediums, formats, and standards. 

Our representatives and your staff have been in close, harmonious contact from the earliest stages of 
planning for the survey, right up until the final stages of review of the final report. I want to take this 
opportunity to thank especially both the former and present Superintendents of Documents, as well as the 
staffs of the present and former directors of the Library Programs Service, and the Office of Electronic 
Information Dissemination Service, for the superb support NCLIS and the contractor received throughout 
the process. 
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I also want to recognize the key role played by Forest Woody Horton, Jr. As consultant to NCLIS, 
Woody’s broad knowledge of how Government works and his deep understanding of Information 
Resources Management helped to move the study along most effectively. 

Finally, I would like to recognize the support of Vice-Chair Martha B. Gould, Commissioners C.E. 
(“Abe”) Abramson who chairs the NCLIS Access to Government Information Committee, Joan R. 
Challinor, and Jose-Marie Griffiths, all of whom have been staunch advocates throughout. I believe you 
are also aware of the strong interest and support NCLIS Executive Director Robert S. Willard personally 
accorded this study, beginning very early with his tenure as a commissioner and extending to the present 
day. 



The long review and analysis process of the contractor’s statistical tabulations, findings, and 
observations has just begun. This demanding process will take some time, in part because the number of 
interested communities is so large, and in part because the subject matter is so technical, involving the full 
range of information handling formats, mediums, and standards, and quite diverse agency plans and 
practices. Ultimately, actions needed to be taken will most likely involve new or strengthened policies, 
rules, and regulations, as well as the adoption of technical standards, some of which could have legislative 
ramifications. 

It is now the Commission’s intention to begin Phase III. We will take the now completed Phase II 
Westat report, as well as the Phase I report completed in 1997 by the National Academy of Sciences, as 
points of departure. They will be reviewed and we will determine if additional fact gathering is required. 
We can then move forward to draw conclusions and make recommendations to the Congress and the 
President from the multitude of facts and expert opinions received thus far. 

NCLIS will continue to consult with GPO, along with various knowledgeable individuals, 
interagency and special advisory groups, all of whom have been assisting us throughout the Phase I and II 
efforts, as we prepare a plan for the Phase III initiative. My hope is that we will keep most of the broader 
advisory team we have utilized thus far in place until we have completed Phase III. 

Finally, I want to thank you for your personal leadership, without which we could have never moved 
ahead with this complex, landmark task. 



Sincerely yours. 






Jeanne Hurley Simon 
Chairperson 
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Executive 

Summary 



The Federal Depository Library Program (FDLP) has served 
and continues to serve the American public by ensuring 
localized access to Federal Government information. The 
mission continues to be as important today to the fundamental 
success of our democracy as it was when the FDLP was created. 
The FDLP's original mandate, to assist Americans regardless 
of economic, education, or geographic considerations, is one 
that must not be lost as we strategically and thoughtfully use the 
tools of the electronic age to enhance that mandate. 

Letter to Michael F. DiMario, the Public Printer, from Senators 
John Warner and Wendell Ford of the Senate Committee on 
Rules and Administration, May 24, 1996. 



Background 



Congress established the antecedents to the Federal Depository Library 
Program (FDLP) in the Act of 1813 to ensure that the American public has 
access to its Government’s information. The mission of the FDLP, part of 
the Superintendent of Documents (SuDocs) in the Government Printing 
Office (GPO), is to assure current and permanent public access to the 
universe of information published by the U.S. Government. Depository 
libraries safeguard the public’s right to know by collecting, organizing, 
maintaining, preserving, and assisting users with information from the 
Federal Government. GPO provides that information at no cost to 
designated depository libraries throughout the country. These depository 
libraries, in turn, provide local, no-fee access to Government information in 
all formats in an impartial environment with professional assistance. Any 
member of the public can visit these depository libraries and use the Federal 
depository collections. 

In order to administer the FDLP, as required by the enabling legislation for 
the program, 44 U.S.C. Chapter 19, the SuDocs is responsible for the 
acquisition, classification, format conversion, dissemination, and 
bibliographic control of tangible and electronic Government information 
products; the inspection of depository libraries; and the continuing 
education and training initiatives that strengthen the ability of depository 
library personnel to serve the public. An emerging new responsibility is to 
ensure that electronic Government information products disseminated 
through the FDLP, or incorporated in the FDLP Electronic Collection, 
remain permanently accessible to the public. LFnder 44 LF.S.C., Sections 
1901-1903, and Office of Management and Budget (0MB) Circular A-130, 
Management of Federal Information Resources, Federal agencies should 
make all their publications in all formats available to SuDocs for 
distribution to depository libraries. 
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This study to assess electronic medium and format standards for the 
creation and dissemination of electronic information products is an essential 
step toward ensuring a successful and cost-effective transition to a more 
electronic FDLP. The three goals of this assessment were to: 

• Identify medium and format standards that are the most appropriate for 
permanent public access; 

• Assess the cost-effectiveness and usefulness of various alternative 
medium and format standards; and 

• Identify public and private medium and format standards that are, or 
could be used for products throughout their entire information life 
cycle, not just at the dissemination or permanent public access stage. 



The Superintendent of Documents will use the results of this work effort to 

continue to plan and implement the transition to a more electronic FDLP. 

The five major specific objectives are: 

• First, with respect to electronic publishing practices and plans for 
Federal agencies (including ways in which the FDLP can best 
accommodate them), the objective is to provide an analysis of current 
practices as well as future plans for creating, disseminating, and 
providing permanent public accessibility to electronic information 
products, and to identify the standards for software and electronic 
mediums and formats that are used throughout the product’s 
information life cycle, from creation to archiving but especially at the 
stage of dissemination for permanent public access. 

• Second, with respect to cost-effectiveness of various dissemination 
mediums and formats that are, or could be utilized, the objective is to 
gather information on standards (whether mandated or consensual) that 
will assist the FDLP in making near-term decisions regarding the cost- 
effectiveness of alternative mediums and formats for all FDLP 
participants. This information should also assist participants in long- 
term planning for permanent public accessibility, and the collection and 
analysis of overall information life cycle costs. 

• Third, with respect to the practical utility of various electronic 
mediums and formats to depository libraries and the public, the 
objective is to identify preferred standards used in various mediums 
and formats that depository libraries will need to support. 

• Fourth, with respect to utilizing standards employed in mediums and 
formats that can be used throughout all stages of the information life 
cycle (including creation, composition, computer terminal display, 
encryption, secure digital signature with non-repudiation, and secure 
transmission capabilities), for electronic dissemination, but especially 
permanent public accessibility, the objective is to assess standards for 
basic security services in order to provide for secure and reliable 
transmission and document interchange. 
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• Fifth, with respect to standards that are being developed and used in the 
private sector, the objective is to identily existing and planned 
standards for the purpose of determining what the FDLP must do to 
accommodate their adoption in terms of hardware/software 
requirements, staff and user education and training, and budgetary 
impacts. 



Methodology 



The study utilized both quantitative and qualitative data collection 
activities; a survey of a cross-section of 314 Government information 
products from 24 agencies and interviews with experts. The response rate 
for the survey was 74 percent. This cross-section of products was not a 
randomly selected sample due to cost and time constraints. Instead, 
NCLIS and GPO — assisted by various groups, including the library 
associations represented by the Inter-Association Working Group on 
Government Information Policy (lAWG), the Federal Library and 
Information Center Committee (FLICC), the Depository Library Council 
(DLC), and the Interagency Council on Printing and Publication Services 
(ICPPS) — developed and refined the criteria for product selection. NCLIS, 
GPO, and the other organizations asked knowledgeable members of these 
groups to identify products that met one or more of six criteria. 

NCLIS distributed the list of preliminary products to agency Chief 
Information Officers (CIOs) who were asked to validate and coordinate the 
final selections with their appropriate agency personnel. In addition, 
NCLIS asked CIOs to select an agency coordinator. The coordinator’s role 
was to oversee the distribution of product questionnaires to the appropriate 
respondents and to encourage respondents to complete the questionnaire and 
return it to Westat. 

Product selection was based on six criteria: 

• Increased emphasis on electronic dissemination, rather than 
continuation of paper and microform dissemination; 

• Replacement of older electronic mediums and formats with state-of-the- 
art technologies; 

• Adoption of mandated (Government or private sector) and consensual 
(common agency practice) medium and format standards; 

• Adoption and use of preferred mediums or formats that have 
widespread support from agency, depository library, and user 
communities; 

• Exemplified cost-effective mediums and standards, especially those that 
can be used throughout the entire information life cycle, rather than the 
use of expensive customized or shelf packages; and 
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• Exemplified awareness of the important impact of medium and format 
decisions on permanent accessibility, authentication, and/or security 
encryption protection. 

The survey requested information on four main topics: 

• General information about the product and agency that produced it. 

• The product’s current profile including the kinds of data the product 
contains, mediums in which it is produced, formats and online 
approaches used (if applicable); and searchability and retrievability of 
the product. 

• Future plans for the product including changes in its data, mediums, 
and formats. 

• Other issues including metadata, permanent public access, permanent 
retention, authenticity, updating/upgrading plans, user fees, licensing, 
and public domain. 

The qualitative data collection included site visits to three depository 
libraries, meetings with representatives of five Government agencies, and 
telephone interviews with six experts. The qualitative data collection 
included site visits, agency meetings, and expert interviews. Westat 
conducted site visits to three Federal depository libraries: 



• McKeldin Library, University of Maryland College Park, College Park, 
Maryland 

• Washington College of Law Library, American University, 
Washington, D.C. 

• Montgomery County Rockville Regional Public Library, Rockville, 
Maryland 

The purpose of the visits was to discuss the effects of the transition to a 
more electronic Federal Depository Library Program on the end user and on 
the services and resources of each library. 

Meetings with agency representatives had a twofold purpose: 

• To collect qualitative data about electronic Government information 
products, such as cost-effectiveness of standards, use of locator tools, 
results of user surveys, etc., that were not covered in the survey; and 

• To discuss the procedures for distribution of the questionnaire. 

In addition to inviting agency coordinators and respondents, the statement 
of work specified that Westat invite representatives of the following offices 
to attend the meetings: 
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• Public affairs or communications offices, 

® Agency printing and publishing units, 

• Information technology or electronic information systems offices, 

• Agency libraries, and 

• Relevant program offices. 

The follovv^ing six agencies agreed to schedule a meeting: Department of 
Health and Human Services, Department of Education, U.S. Supreme 
Court, Department of Commerce, Environmental Protection Agency, and 
the National Archives and Records Administration. Only four of the six 
agencies chose to discuss the qualitative questions at the meeting. The other 
two agencies discussed the questionnaire only and agreed to respond to the 
discussion questions in writing, although only one actually submitted their 
written questions. 

Finally, Westat held four telephone interviews with six content experts. 
The experts included two webmasters (Linda Wallace from the Internal 
Revenue Service, and Jerry Malitz from the National Center for Education 
Statistics); two preservation specialists (Evelyn Frangakis from the 
National Agricultural^ Library, and Abby Smith from the Council on 
Library and Information Resources); and two professors in information 
resources management (John Bertot and Charles McClure). The purpose of 
expert interviews was to: 

• Solicit opinions of experts on topics not adequately covered on the 
survey or in the agency meetings, 

• Ask questions to provide a broader context in which to view the issues, 
and 

• Explore current initiatives and future directions. 



Key Findings 



These findings reflect the major results of the survey and qualitative data 
collection: 



Policy and Planning Issues 

1. There is an overall lack of Government information policy guiding 
electronic publishing, dissemination, permanent public access, or 
information life cycle management, especially as information policy 
relates to agency missions. Also, there is a lack of overall coordination 
of these initiatives at the Governmental, branch, or even agency level 
(pp. 68-69). 

2. Responsibility for electronic publishing within agencies is decentralized, 
diffuse, and unclear. Some agencies either could not 
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identify or had difficulty identifying the proper respondent within their 
own agency, or even the person who was responsible for the product 
(pp. 1 1 and 14). 

3. Some Government agencies are monitoring the information needs of 
their users to enhance current access to electronic Government 
information products (p. 65). 

4. There is a lack of specific planning for product development and 
technological migration (pp. 34-36; table 23 on p. 42). 

5. There is a lack of planning for or consideration of web design 
approaches that comply with the Americans with Disabilities Act 
(ADA) (table 6a, p. 29). 

Permanent Public Access 

6. The concept of permanent public access (PPA) is not well understood. 
Respondents also had difficulty distinguishing between PPA for 
electronic products and archiving electronic Federal records with the 
National Archives and Records Administration (tables 18-20, pp. 39- 
40). 

7. Metadata and their importance to public access are not well understood, 
particularly as they may affect PPA. Only 27 percent of respondents 
reported having a metadata record for the products surveyed (table 19, 
p. 39). 

8. For some products, PPA results from the agencies’ use of a host 
disseminator, such as GPO Access (p. 1 1). 

Authenticity 

9. There is a lack of understanding of what ensuring authenticity entails, 
and a lack of planning for or consideration of ensuring authenticity of 
electronic Government information products (table 21, p. 41). 

Product Characteristics 

10. Fifteen percent of the products surveyed are not in the public domain, 
for all or part of the product (table 27, p. 45). In addition, user fees are 
charged for 30 percent of the products (table 24, p. 43). 

1 1 . The most prevalent types of mediums are the web, paper, CD-ROM, 
and bulletin board systems (table 3a, p. 22); the most prevalent formats 
are HTML, PDF, GIF, JPEG, TIFF, and ASCII (table 4a, p. 25). 

12. The most prevalent types of data contained in the products surveyed are 
textual, numerical, bibliographic, and graphical (tables 2a and 2b, p. 
20 ). 
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Standards 

13. There is a lack of standardization for producing Government 
information products on CD-ROM (e.g., installation instructions, user 
documentation) (p. 55). 

14. The most prevalent medium and format standards identified in the 
survey are common agency practice rather than agency-mandated 
(tables 3b, 4b, 6b, pp. 23, 26, and 30). 

15. Some Government agencies have established guidelines or best 
practices for presenting and organizing Government information 
products on the web, although full compliance with the guidelines is a 
goal that has not yet been achieved (p. 64). 

1 6. Some Government agencies are exploring a range of innovative formats 
and web design approaches for electronic Government information 
products (p. 57). 



Next Steps 



As a followup effort, NCLIS indicated that they will use these findings as a 
point of departure and analyze them in greater depth. It is expected that this 
followup effort will result in broad conclusions and recommendations to the 
President and Congress about how the problems and challenges revealed in 
this study can be constructively addressed to improve current and future 
public access to electronic Government information. 
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1 Introduction 
and 

Background 



Since 1813, the American public has benefited from the ability to gain free 
access to Federal Government information. This unique American right to 
no-fee access to Government information is made possible through the 
Federal Depository Library Program (FDLP) of the Superintendent of 
Documents (SuDocs) in the Government Printing Office (GPO). The 
FDLP has significantly contributed to creating an informed, educated, and 
culturally enriched U.S. citizenry. 



This introduction provides a brief overview of the FDLP and background 
information on the purpose and objectives of this study to assess electronic 
Government information products. 



The Federal 
Depository 
Library Program 



The mission of the Federal Depository Library Program is to assure current 
and permanent public access to the universe of information published by the 
U.S. Government. The FDLP was established by Congress to ensure that 
the American public has access to its Government’s information. 
Depository libraries safeguard the public’s right to know by collecting, 
organizing, maintaining, preserving, and assisting users with information 
from the Federal Government. The Government Printing Office provides 
Government information at no cost to designated depository libraries 
throughout the country. These depository libraries, at their own expense, 
provide local, no-fee access to Government information in all formats in an 
impartial environment with professional assistance. Any member of the 
public can visit these depository libraries and use the Federal depository 
collections. 



Products distributed by GPO for depository library collections include all 
electronic Government information products that are of public interest or 
educational value. By law, the FDLP excludes those products that are 
solely for administrative or operational purposes, classified for reasons of 
national security, or the use of which is constrained by privacy 
considerations (GPO, 1998, p. 4). 

In order to administer the FDLP, as required by the enabling legislation for 
the program, 44 U.S.C. Chapters 17, 19, and 41, the SuDocs is responsible 
for the acquisition, classification, format conversion, dissemination, and 
bibliographic control of tangible and electronic Government information 
products; the inspection of depository libraries, and the continuing 
education and training initiatives that strengthen the ability of depository 
library personnel to serve the public. An emerging new responsibility is to 
ensure that electronic Government information products disseminated 
through the FDLP, or incorporated in the FDLP Electronic Collection, 
remain permanently accessible to the public. Under 44 U.S.C., Sections 
1901-1903, and Office of Management and Budget (0MB) Circular A-130, 
Management of Federal Information Resources, Federal agencies should 
make all their publications in all produced formats available to SuDocs for 
distribution to depository libraries. 




21 



1 



Assessment of Electronic Government Information Products 



How the Federal 
Depository Library 
Program Works 



Background 
of the Study 




GPO provides Government information at no cost to designated depository 
libraries throughout the country. These depository libraries, at their own 
expense, provide local, no-fee access with professional assistance to this 
information in all formats. Access to Federal Government information is 
available through more than 1,350 depository libraries located throughout 
United States and its territories. Fifty-three of the depositories are regionals, 
and the remaining are selective depositories. The regional libraries receive 
and maintain everything that is distributed through the program, unless they 
are superseded. The selective libraries pre-select the types of publications 
they wish to receive based on the specific needs and interests of the 
communities they serve. Of the libraries in the FDLP, approximately 50 
percent are academic, 20 percent are public, 1 1 percent are law, 5 percent 
are community college, 4 percent are Federal agency, and 10 percent are 
special, state, court, and Federal court libraries. 

Before the evolution of electronic publishing media, especially the Internet, 
Federal Government agencies published information almost exclusively in a 
centralized print environment that facilitated easy distribution to the Federal 
depository libraries. Now, Federal Government agencies are doing their 
own electronic publishing and creating and managing their own websites to 
disseminate a variety of Government information products. This study 
resulted from Congress's concerns about the short- and long-term effects of 
electronic publishing on the ability of all U.S. citizens to continue to gain 
affordable and easy access to Government information. 



This study to assess electronic Government information products was 
authorized by the Joint Committee on Printing and was sponsored by the 
Superintendent of Documents, U.S. Government Printing Office. The 
initial need for this project was identified in GPO’s cooperative 1996 Study 
to Identify Measures Necessary for a Successful Transition to a More 
Electronic Federal Depository Library Program. This study (see 
www.access.gpo.gov/su_docs/dpos/fdlppubs. html#4) was conducted at the 
direction of Congress. In order to conduct the study, the Public Printer 
established a working group consisting of representatives from the 
following program stakeholders and constituents: 

• GPO, 

• Appropriate congressional committees, 

• Congressional Research Service at the Library of Congress, 

• Office of Management and Budget, 

• National Archives and Records Administration, 

• Federal Publishers Committee, 
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Project Phases 



• Interagency Council on Printing and Publication Services (ICPPS), 

• Administrative Office of the U.S. Courts, and 
» Depository library community. 



One of the committee's major recommendations was to assess electronic 
medium' and format standards for the creation and dissemination of 
electronic information products. The committee considered this assessment 
an essential step toward ensuring a successful and cost-effective transition 
to a more electronic FDLP. 



This project is being undertaken in three phases. The first phase of the 
project consisted of a review by the National Academy of Science’s 
Computer Science and Telecommunications Board (CSTB) in which CSTB 
developed a detailed statement of work that defined the data collection 
process required to conduct the assessment (see 
http://www.nclis.gov/info/gpol.html). 

This report is a product of Phase II of the project. GPO commissioned the 
National Commission on Libraries and Information Science (NCLIS) to 
undertake a survey and assessment of electronic Government information 
products. NCLIS awarded the contract to Westat, a survey research 
company, to undertake research and data collection from Federal agencies 
in all three Branches, as well as solicit the opinions of selected 
knowledgeable experts. The contract further called for Westat to complete 
an analysis of the data and expert opinions for the purpose of interpreting 
their general meaning and significance, including identifying broad 
emerging trends and patterns, and documenting findings. 

In Phase III, NCLIS will identify an appropriate organization to review 
Phase I and Phase II findings, as well as to review the data and develop 
conclusions and recommendations for GPO, the Congress, and the 
President. 



' The word medium is used throughout this report. It refers to the physical, chemical, or biological 
substrate used to create, organize, store, search for, retrieve, disseminate, or permanently archive data, 
documents or literature, including paper, microforms, fiber optic cables, photographic film, CD-ROMs, 
floppy diskettes, magnetic storage devices, sound recordings, and videotape. The term media is not used 
in this report as the plural of “medium.” Media is the means used to publish, communicate, disseminate, 
and distribute information, regardless of format, such as radio, television, magazines, office or home 
PCs, scholarly journals, and videotape. 
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Study Goals and 
Objectives (Phase II) 



• Identify medium (see glossary in Appendix E for the difference between 
the medium and media) and format standards that are the most 
appropriate for permanent public access, 

• Assess the cost-effectiveness and usefulness of various alternative 
medium and format standards, and 

• Identify public and private medium and format standards that are, or 
could be used for products throughout their entire information life 
cycle, not just at the dissemination or permanent public access stage. 

The Superintendent of Documents will use the results of this work effort to 

continue to plan and implement the transition to a more electronic FDLP. 

The five major specific objectives are: 

• First, with respect to electronic publishing practices and plans of 
Federal agencies (including ways in which the FDLP can best 
accommodate them), the objective is to provide an analysis of current 
practices as well as future plans for creating, disseminating, and 
providing permanent public accessibility to electronic information 
products, and to identify the standards for software and electronic 
mediums and formats that are used throughout the product’s 
information life cycle, from creation to archiving, but especially at the 
stage of dissemination for permanent public access. 

• Second, with respect to cost-effectiveness of various dissemination 
mediums and formats that are, or could be utilized, the objective is to 
gather information on standards (whether mandated or consensual) that 
will assist the FDLP in making near-term decisions regarding the 
cost-effectiveness of alternative mediums and formats for all FDLP 
participants. This information should also assist participants in 
long-term planning for permanent public accessibility, and the 
collection and analysis of overall information life cycle costs. 

• Third, with respect to the practical utility of various electronic 
mediums and formats to depository libraries and the public, the 
objective is to identify preferred standards used in various mediums 
and formats that depository libraries will need to support. 

• Fourth, with respect to utilizing standards employed in mediums and 
formats that can be used throughout all stages of the information life 
cycle (including creation, composition, computer terminal display, 
encryption, secure digital signature with non-repudiation and secure 
transmission capabilities), but especially for permanent public 



Information gathered from this assessment will be used by the 
Superintendent of Documents to facilitate improved public access to 
Federal Government information made available to Federal depository 
libraries and the general public through the FDLP. More specifically, for 
this cross-section of Government information products, the Phase II goals 
were to: 
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accessibility, the objective is to assess standards for basic security 
services in order to provide for secure and reliable transmission and 
document interchange. 

• Fifth, with respect to standards that are being developed and used in the 
private sector, the objective is to identify existing and planned 
standards for the purpose of determining what the FDLP must do to 
accommodate their adoption in terms of hardware/software 
requirements, staff and user education and training, and budgetary 
impacts. 



Scope and 
Organization 
of the Report 



The primary data collection activities included a survey and interviews. 
Westat, per the requirements established by NCLIS in consultation with 
GPO, surveyed a cross-section of electronic information products from 
Federal agencies in all three branches of Government and solicited the 
opinions of selected knowledgeable experts. This cross-section of products 
was not a randomly selected sample due to cost and time constraints. 
Therefore, readers are cautioned about generalizing the findings to all 
electronic Government information products. 

Westat surveyed electronic Government information products to determine 
the mediums and formats in which products are currently produced and the 
standards, if any, that are being used. The survey also asked respondents 
questions about the agency’s future plans for adding or changing products, 
including the mediums and formats in which they will be disseminated for 
permanent public access. 

This report is limited to presenting and discussing the survey findings and 
findings from qualitative site visits, agency meetings, and expert interviews. 
Phase III of the project will focus on drawing conclusions and 
recommendations based on work conducted during Phases I and II. 

The report is organized in five parts; introduction and background, 
methodology, survey analysis and findings, qualitative findings, and 
discussion of quantitative and qualitative findings. Please note that 
Appendix E contains a glossary of terms and acronyms used on the 
questionnaire and throughout this report. 
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Methodology the report discusses the following topics: 

• The process of selecting a cross-section of electronic Government 
information products, 

• Agency coordinator briefings, 

• Questionnaire design and development, 

• Nonresponse and data retrieval followup, and 

• The methodology for the qualitative data collection activities, i.e., site 
visits, agency meetings, and expert interviews. 



Product Selection NCLIS and GPO — assisted by various groups, including the library 

associations represented by the Inter-Association Working Group on 
Government Information Policy (lAWG), the Federal Library and 
Information Center Committee (FLICC), the Depository Library Council 
(DLC), and the Interagency Council on Printing and Publication Services 
(ICPPS) — developed and refined a set of criteria for product selection. 
NCLIS, GPO, and the other representatives asked knowledgeable members 
of these groups to identify products that met one or more of the following 
six guidelines: 

• Increased emphasis on electronic dissemination rather than continuation 
of paper and microform dissemination; 

• Replacement of older electronic mediums and formats with state-of-the- 
art technologies; 

• Adoption of mandated (Government or private sector) and consensual 
(common agency practice) medium and format standards; 

• Adoption and use of preferred mediums or formats that have 
widespread support from agency, depository library, and user 
communities; 

• Exemplified cost-effective mediums and standards, especially those that 
can be used throughout the entire information life cycle, rather than the 
use of expensive customized or shelf packages; and 

• Exemplified awareness of the important impact of medium and format 
decisions on permanent accessibility, authentication, and/or security 
encryption protection. 



The products were not randomly selected; therefore, readers are cautioned 
about generalizing the findings to all electronic Government information 
products. 

In April 1 998, NCLIS distributed the preliminary list of products to agency 
Chief Information Officers (CIOs), who were asked to validate and 
coordinate the final selections with appropriate agency personnel. In 
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addition, NCLIS asked CIOs to select an agency coordinator. The 
coordinator’s role was to oversee the distribution of product questionnaires 
to the appropriate respondents and to encourage respondents to complete 
the questionnaire and return it to Westat. (See Appendix B for a list of 
coordinators who participated in this study.) 

The final product list included 328 products from 24 agencies (Appendix 
C). Over the course of the data collection, the number of products 
decreased from 328 to 3 14 for the following reasons: 

• Several products were discontinued and no longer exist. 

• Several products were in paper only and agencies had no plans to 
migrate them to an electronic medium; therefore, they fell outside the 
scope of this study. 

• Agency coordinators could not identify respondents for some products, 
so there was no one to complete the questioimaire. 

• Several questioimaires were undeliverable due to unknown or incorrect 
respondent addresses; no alternate respondent could be located in a few 
cases. 



Coordinator 

BrieHngs 



NCLIS and GPO plaimed and conducted two coordinator briefings in June 
and July 1998, and asked Westat to attend them (see Appendix A for 
agenda). The purpose of these briefings was to: 

• Provide an overview of the study including background, purpose, goals, 
and schedule, 

• Discuss their specific tasks, 

• Review the draft questioimaire with coordinators and solicit their input 
on changes, 

• Collect their final list of products, and 

• Thank them for their participation and cooperation. 

Coordinators were asked to: 

• Assist Westat in pretesting the survey instrument, 

• Identify and brief appropriate internal participating offices, 

• Identify product respondents for survey followup, 

• Schedule and participate in voluntary agency meetings with Westat, 

• Distribute questionnaires to agency respondents, 

• Ensure timely completion and submission of survey instruments, and 

• Cooperate with Westat on followup. 
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Only a few coordinators brought their final selections to the agency 
meetings; most agencies needed much more time to review and finalize their 
product selections. The questionnaire review also served as an informal 
pretest of the questionnaire. 



Questionnaire 

Design 



NCLIS, with consultation from GPO, developed the initial five-page list of 
questions. This list of questions was included as an appendix to the 
statement of work. Westat worked with GPO and NCLIS from June 
through July to expand and refine the list of questions to a 13-page 
instrument with appropriate instructions, examples, skip patterns, open- 
ended questions, please-specify questions, etc. Westat pretested the 
questionnaire informally at the two coordinator briefings. The coordinators 
helped Westat to clarify some questions, expand the format choices, and 
add a few more questions. 

Westat conducted a more formal pretest with personnel from six 
Government agencies. These pretests led to the following substantive 
changes in the questionnaire: 

• Clarification of instructions and wording of several questions, 

• Addition of more format options, 

• Addition of the definition of “product” at the beginning of the 
questionnaire, and 

• Clarification of definitions included in the glossary. 

Westat, with final approval by NCLIS and GPO, finalized the questionnaire 
by mid- August 1998. (See Appendix D for the cover letters and Appendix 
E for the final questionnaire.) 



Distribution of the 
Questionnaires 



During the last week of September and the first week of October, Westat 
distributed the questionnaires to 23 agencies through the agency 
coordinators. On October 9, 1998, NCLIS requested that Westat add more 
products to the survey by including a 24th agency, the Securities and 
Exchange Commission (SEC). 



Westat created a database of products and corresponding coordinators or 
respondents and their addresses, phone and fax numbers, and e-mail 
addresses, and prepared and mailed packets to the agency coordinators. The 
agency coordinators were responsible for ensuring that each packet was 
sent to the appropriate product respondent in a timely fashion. These 
packets included the following materials for each product that was to be 
surveyed: 



• Cover letter to coordinator. 
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• Cover letter to respondent, 

• Questionnaire, 

• Glossary of terms used in the questionnaire, and 

• Postage-paid return envelope. 

A few agency coordinators requested that Westat send questionnaires 
directly to their product respondents and a copy of the respondents' packets 
to the coordinators themselves. Westat sent questionnaire materials directly 
to the respondents at the Department of Commerce, the Department of the 
Interior, the Executive Office of the President, and the U.S. Congress. 
These respondent packets included the following materials: 

• Cover letter to respondent, 

• Questionnaire for each product he/she was assigned to survey, 

• Glossary of terms used in the questionnaire, and 

• Postage-paid return envelope. 

In addition, the Department of Health and Human Services asked Westat to 
send an e-mail message to the individual product respondents notifying them 
that they could download the final version of the questionnaire and cover 
letters from the PDF file located on the NCLIS website at 
http://www.nclis.gov/news/nclisqux.pdf in order to complete the 
questionnaire. 



Followup for 
Nonresponse, Data 
Retrieval, and 
Inconsistency 



Westat made the first calls for nonresponse to agency coordinators. These 
calls began in early November and continued through mid-December. In 
addition, NCLIS sent periodic coordinator bulletins to keep coordinators 
updated on the progress of the study and to encourage respondents — 
through the coordinators — to complete questionnaires and return them to 
Westat. 



Westat began a second round of nonresponse followup calls to respondents 
from mid-December through the end of January 1999. From mid- 
November through the first week in January 1999, Westat made calls 
directly to respondents for data retrieval (i.e., missing data) and 
inconsistencies (i.e., a respondent checked “yes” to one question, but the 
next question was answered in a way that suggested a “no” answer to the 
first question). 

Approximately 40 percent of the questionnaires required some type of data 
retrieval followup for one or more questions. Some questions, such as 16, 
18-19, and 21a, concerning metadata, permanent retention. 
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authenticity, and the product’s supporting technology, presented particular 
problems. Westat added a “don’t know” category to these questions as a 
result of the nonresponse data retrieval. In addition, most respondents 
skipped questions 13d, 14d, and 15d about long-term plans for changing the 
product. Data retrieval phone calls and discussions with agency 
coordinators suggest respondents skipped these questions because agencies 
had not yet developed long-term plans. 

The calls to respondents for data retrieval and data inconsistency revealed 
the following reasons for nonresponse: 

• Did not know the answer. 

• Could not identify anyone who knew the answer. 

• Did not understand the question or the concept; using glossary did not 
help. 

• Did not have time to research the answer; had other work priorities. 

In a few instances, it was clear that the agency was not in a good position to 
respond to the questionnaire, in part because they rely on another agency, 
vendor, or contractor to provide electronic access to their products. 
Sometimes these “host disseminators,” such as GPO, assisted in preparing 
the responses sent in by the publishing entity. 

Observations about the data collection process. Agency coordinators 
had difficulty locating a single point of contact from each agency sub-unit 
who was knowledgeable about the range and type of electronic information 
products created for the agency. Furthermore, due to the nature of the 
survey questions, product respondents had to coordinate responses to some 
questions with personnel who often did not work in their program areas. 
This process required respondents to identify personnel with whom they 
appeared to have little prior contact, such as records managers, information 
technology staff, and staff in planning offices, in order to respond to these 
questions. In some cases, this extra step discouraged respondents from 
seeking answers to these questions, so questions were left unanswered. 
Also, agencies whose coordinators could not attend the coordinator 
briefings and agencies that did not participate in the agency meetings had 
more problems with data consistency than did other agencies. 



Methodology for 
Qualitative Data 
Collection 



Site Visits to 
Depository Libraries 



The qualitative data collection included site visits, agency meetings, and 
expert interviews. Westat conducted site visits to Federal depository 
libraries from July 30 through September 9, 1998. The statement of work 
(Appendix J) specified that Westat visit three libraries: one regional 

academic, one law, and one public. Furthermore, GPO suggested that 
Westat visit the following specific libraries in the Washington, D.C., 
metropolitan area: 
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• McKeldin Library, University of Maryland College Park, College Park, 
Maryland 

• Washington College of Law Library, American University, 
Washington, D.C. 

• Montgomery County Rockville Regional Public Library, Rockville, 
Maryland 

The purpose of visits was to discuss the effects of the transition to a more 
electronic Federal Depository Library Program on the end user and on the 
services and resources of each library. The interview questions, which were 
based on readings and discussions with GPO and NCLIS, covered three 
broad areas: 

• What key issues or concerns do you have about users accessing and 
using electronic Government information products? 

• What are your concerns about providing access to electronic 
Government information products? 

• What specific ideas do you have for improving public access to online 
and electronic Government information products in your library? 

The site visits were audiotaped. In addition, the libraries gave Westat 
representatives a tour of the facilities. (See Appendix F for a list of the 
specific interview questions, the names of all interviewees, and detailed site 
visit notes.) 

Site visit observations. In addition to the small number of libraries visited, 
the problems and concerns of librarians in the D.C. metropolitan area may 
not be representative of those experienced by librarians at most depository 
libraries, especially the selective depositories. Some smaller selective 
depository libraries that are located in more remote areas as well as some of 
the larger urban selective depositories might have fewer resources (e.g., 
fewer computers and trained librarians, training funds, and options for low- 
cost Internet providers). 



Purpose and 
Procedures for 
Agency Meetings 



Meetings with agency representatives were held between September 1 5 and 

September 24, 1998. The purpose of the meetings was twofold: 

• To collect qualitative data about electronic Government information 
products that were not covered in the survey, such as cost-effectiveness 
of standards, use of locator tools, results of user surveys, etc.; and 

• To discuss the questionnaire and data collection procedures for 
distribution of the questionnaire. 
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In addition to inviting agency coordinators and respondents, the statement 
of work specified that Westat invite representatives of the following offices 
to attend the meetings: 

• Public affairs or communications offices, 

• Agency printing and publishing units, 

• Information technology or electronic information systems offices, 

• Agency libraries; and 

» Relevant program offices. 

Westat wrote the procedures for scheduling agency meetings and arranging 
for logistics, which included developing meeting protocols, agenda, cover 
letter, and script for interviewers to schedule meetings. We then contacted 
coordinators and sent them the following materials: 

• Cover letter explaining purpose of meeting and their tasks, 

• Meeting agenda and discussion questions, 

• Press release from NCLIS with background information on the project, 

• Roster of potential agency representatives who will attend meeting (to 
be completed by the coordinator), and 

• Respondent product roster (to be completed by coordinator). 

Agency meetings held. Westat contacted 15 of the 24 agencies to hold 
meetings. Of the 7 agencies that were not contacted, 3 had fewer than 10 
products. NCLIS instructed Westat not to hold meetings with the U.S. 
Congress and the Executive Office of the President because NCLIS and 
GPO worked with them directly. 

Ten of the 16 agencies did not respond to Westat’s request to schedule a 
meeting. The following six agencies agreed to schedule a meeting: 

• Department of Health and Human Services 

• Department of Education 

• U.S. Supreme Court 

• Department of Commerce 

• Environmental Protection Agency 

• National Archives and Records Administration 

Only four of the above six agencies chose to discuss the qualitative 
questions at the meeting. The other two agencies wanted to discuss the 
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questionnaire only and agreed to respond to the discussion questions in 
writing. However, only one of them sent in responses. 

Westat audiotaped all agency meetings and took notes as agency personnel 
discussed the questions. (Appendix G includes the list of agencies that 
participated in meetings, the number of attendees, the discussion questions, 
and summary notes from the meetings.) 

In addition to the meetings held with Westat, NCLIS and/or GPO 
representatives met with approximately 50 agency representatives. In these 
meetings, NCLIS and GPO discussed survey goals and objectives and the 
process for preselecting products, in addition to responding to specific 
questions about the survey. 

Agency meeting observations. Agency participation in the entire project 
was voluntary but essential. As with any voluntary activity, participation is 
based on availability and timing. For example, many agency coordinators 
were unavailable to schedule meetings during the summer months, or they 
were available but product respondents were on vacation, which may have 
resulted in fewer agency meetings. 

Product respondents needed to attend the agency meetings to review the 
questionnaire, although they were not always the most appropriate 
personnel to respond to all of the qualitative questions. The project 
depended upon the good faith, interest, and cooperation of agency CIOs and 
coordinators to participate in the meetings. Respondents and participants 
from the private sector are often given an honorarium for participating in 
similar research activities, but Federal employees are exempt from this 
process. 

Scheduling agency meetings, calling coordinators, and preparing paperwork 
to send to coordinators took a considerable amount of planning and 
coordination and time, but it did not result in many meetings. Agencies 
were cooperative, but it was difficult for them to identify the “right” 
personnel to invite to the meetings, even though coordinators took a 
significant amount of time to locate product respondents from other sub- 
units within their agencies. Therefore, answers to the agency meeting 
discussion questions reflected the perspectives of only 5 of the 24 agencies 
surveyed. 



Expert Interviews 



NCLIS provided a list of experts from which Westat chose six names. 
Westat held four telephone interviews with the six experts between October 
27 and November 24, 1998. The experts included two webmasters, two 
preservation specialists, and two professors in information resources 
management. The purpose of expert interviews was to: 
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• Solicit opinions of experts on topics not adequately covered on the 
survey or in the agency meetings, 

• Ask questions raised during the agency meetings or site visits that 
require further explanation, or to provide a broader context in which to 
view the issues, and 

• Explore current initiatives and future directions. 

As with the site visits and agency meetings, Westat audiotaped the 
interviews. Appendix H provides a list of experts, interview questions, and 
a summary of interview notes. 
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Survey 
Analysis 
and Findings 



This section of the report presents the survey findings from each of the 
major survey questions as they appear in the questionnaire (Appendix E). 
Appendix E also includes a glossary of terms and acronyms used 
throughout this report. The discussion and presentation will then focus on 
the key study questions explored on the following topics: 



• Preferred medium and formats used, 

• Planned medium and format changes, 

• Permanent public access issues, 

• Permanent retention issues, 

• Authenticity, and 

• Searchability, proprietary software, and licensing fees. 



The final response rate was 74 percent. Respondents from 24 Government 
agencies completed and returned a total of 242 of the 328 questionnaires 
fielded. The word “respondents” refers to the 242 agency personnel who 
completed the questionnaire. Since each agency submitted at least two 
product questionnaires, the unit of analysis is the product or product 
respondent, not the agency (table 1). The sample was not randomly selected 
due to cost and time constraints. Therefore, readers are cautioned about 
generalizing the findings to all electronic Government information products. 



Structure of the 
Questionnaire 



The questionnaire is organized into five sections, A through E. Section A 
contains general information about the product and agency that produced it. 
Section B contains questions about the product’s current profile including 
the kinds of data the product contains, mediums in which it is produced, 
and, if in an online medium, formats and online approaches used. This 
section concludes with questions on searchability and retrievability of the 
product. Section C relates to the future plans for the product and is 
designed to solicit information about changes in the product’s data, 
mediums, and formats. Section D addresses the issues of metadata, 
permanent public access, permanent retention, authenticity, 
updating/upgrading plans, user fees, licensing, and public domain. The 
final section, E, includes one open-ended general comments question. 
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Section A Responses 



Sections A and B of the questionnaire focus on format and medium 
standards that address the key objectives of the study. Section A contains 
general information about the product and the agency that produced it, 
including the name of the agency and its sub-unit, the product name and 
description, and the Uniform Resource Locator (URL) for the site in which 
the product appears. A list of the agencies surveyed and the number of 
product questionnaires received from each agency appears in table 1. (For 
a description of how products were selected, refer to the methodology 
section.) Appendix C contains the final list of products surveyed. 



Table 1. 




Number of surveys returned by each agency surveyed 


Agency 


Number of 


surveys returned 



Administrative Office of the U.S. Courts 5 

Department of Agriculture 19 

Department of Commerce 14 

Department of Defense 8 

Department of Education 14 

Department of Energy 12 

Department of Health and Human Services 19 

Department of the Interior 1 1 

Department of Justice 8 

Department of Labor 2 

Department of State 3 

Department of Transportation 9 

Department of the Treasury 13 

Environmental Protection Agency 16 

Executive Office of the President 5 

General Services Administration 8 

Library of Congress 2 1 

National Aeronautics and Space Administration 6 

National Archives and Records Administration 10 

Securities and Exchange Commission 1 1 

Smithsonian Institution 1 1 

Social Security Administration 4 

Supreme Court of the United States 4 

United States Congress 9 



SOURCE: National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire: 1998. 
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Section B Responses 



er|c 



Section B covers the current product profile, including: 

• How it is used; 

• What types of data it contains; 

• What mediums the product is available in, what is the primary medium 
used, and what are the agency’s medium standards; 

• What kinds of formats are used, what is the primary format used, and 
what are the agency’s format standards; 

• What user interfaces are supported and what web design approaches 
are used; 

• If the electronic product can be searched and how; 

• What agency hosts the product on the web; and 

• How the product can be retrieved. 

Readers should note that most of the survey questions asked respondents to 
“check all that apply”; therefore, the percentages for these questions will 
exceed 100 percent. Also, for the first set of tables in this section (tables 1 
through 6), the response categories appear in descending order by number 
or percentage. Therefore, the responses will not match the order in which 
they appear on the questionnaire. 



Types of Data Contained in Product 

Table 2a shows that the frequently mentioned types of data contained in the 
products surveyed are textual (188 responses), followed by graphical (142 
responses), numerical (141 responses), bibliographic (82 responses), and 
spatial (53 responses). Multimedia, video, and sound are less common, 
probably because they reflect the products surveyed and because of the 
special plug-ins, hardware, and memory required to open, view, and listen 
to products that contain these data types. The primary data types contained 
in products surveyed are textual (57 percent), numerical 

(21 percent), bibliographic (10 percent), and graphical (5 percent; 
table 2b). These four types of data account for approximately 93 percent of 
the products surveyed. 
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Table 2a. 






Number and percent of types of data, by the type of data contained 


Type of data 


Type of data contained 


Number 


Percent 



Textual data (books, serials, reports) 188 111 

Graphical data (photos, charts, graphs, tables, drawings) 142 58,7 

Numerical data 141 58,3 

Bibliographic data 82 33,9 

Spatial data (maps, coordinate files) 53 21.9 

Multimedia (sound, video, text, graphics) 14 5,8 

Video 10 4,1 

Sound 9 3,7 

Other 16 6,6 



NOTE: Percents do not add to 100 because respondents could choose more than one item. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire, 1998, 



Table 2b. 

Number and percent of types of data, by the primary type of data 



Type of data 



Primary type of data 



Number 



Percent 



Textual data (books, serials, reports) 138 57.0 

Numerical data 50 20.7 

Bibliographic data 24 9.9 

Graphical data (photos, charts, graphs, tables, drawings) 13 5.4 

Multimedia (sound, video, text, graphics) 3 1.2 

Spatial data (maps, coordinate files) 2 0.8 

Sound 1 0,4 

Video 1 0,4 

Other 10 4J_ 

NOTE: Percents do not add to 100 because of rounding. 



SOURCE: National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire, 1998. 



Types of Mediums Used 

Respondents were asked to identify all the types of mediums in which the 
product is available to the public as well as the primary type of medium 
used. The most common type of medium used among pre-electronic 
mediums is paper (177 responses), followed by microform (22; table 3a). 
The responses in the “other” category include Fax on Demand, audiotapes, 
and Braille. Among electronic mediums used, it is not surprising that the 
web is the most common (204 responses), followed by 
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CD-ROM (70 responses), floppy diskettes (42 responses), hard drive 
(30 responses), and magnetic tape (18 responses). These figures reflect the 
medium types the public is most likely to easily access, as well as the 
availability and growing interest in the web. 

Table 3a also displays the frequency and percentage distribution of the 
primary types of mediums in which the product is publicly accessible. The 
web (42 percent) and paper (4 1 percent) are the primary types of mediums 
used, followed by CD-ROM (8 percent) as a distant third. 

Standards for all mediums checked. For each type of medium checked, 
respondents identified one medium standard (see Appendix E glossary) 
among four types: 

• Agency mandated, 

• Common agency practice, 

• Other, and 

• None. 



While most agencies have some type of standards for their pre-electronic 
and electronic mediums, they are primarily “common agency practice” 
rather than “agency mandated.” For pre-electronic mediums, 33 percent of 
the products in paper are in an agency-mandated standard (table 3b). 
However, 52 percent of paper products are used as a common agency 
practice. Only 13 percent of the CD-ROM products are in an agency- 
mandated standard, as compared to 59 percent of CD-ROMs that are used 
as a common agency practice. Eighteen percent of web-based products 
were reported to be in an agency-mandated standard, while 70 percent of 
them are used as a common agency practice. 

A considerable number of products in CD-ROM (21 percent) were reported 
as having no standards (table 3 b). Compare these numbers to 
9 percent (15 products) of products reported by respondents as having no 
standards for the use of paper, and 8 percent (16 products) reported as 
having no standards for the use of the web. 




39 



21 



Assessment of Electronic Government Information Products 



Table 3a. 

Number and percent of mediums publicly available, by the type of medium used and primary medium 
used 



Medium 


Type of medium used* 


Primary medium used^ 


Number 


Percent 


Number 


Percent 


Online mediums 










Web 


204 


84.3 


102 


42.1 


Bulletin Board Systems 


13 


5.4 


1 


0.4 


Gopher 


10 


4.1 


0 


0.0 


Other 


24 


9.9 


3 


1.2 


Pre-electronic mediums 


Paper 


177 


73.1 


99 


40.9 


Microform 


22 


9.1 


0 


0.0 


Other 


12 


5.0 


2 


0.8 


Optical mediums 


CD-ROM 


70 


28.9 


20 


8.3 


WORM (write once, read many disk) 


1 


0.4 


1 


0.4 


DVD (digital video disk) 


1 


0.4 


0 


0.0 


Other 


1 


0.4 


1 


0.4 


Electronic mediums 


Floppy diskette 


42 


17.4 


1 


0.4 


Hard drive 


30 


12.4 


4 


1.7 


Magnetic tape 


18 


7.4 


1 


0.4 


Other 


10 


4.1 


1 


0.4 



‘Column percents do not add to 1 00 because respondents could choose more than one item. 

^Percents do not add to 100 due to nonresponse. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product Assessment Questionnaire, 1998. 
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Table 3b. 

Number and percent of mediums publicly available, by the standard for each medium used 



Standard for each medium used 



Medium 


Agency mandated 


Common agency 
practice 


Other 


None 




Number 


Percent 


Number 


Percent 


Number 


Percent 


Number 


Percent 



Online mediums 



Web 


36 


17.6 


143 


70.1 


9 


4.4 


16 


7.8 


Bulletin Board Systems 


1 


7.7 


10 


76.9 


0 


0.0 


1 


7.7 


Gopher 


0 


0.0 


8 


80.0 


0 


0.0 


2 


20.0 


Other 


4 


16.7 


14 


58.3 


0 


0.0 


6 


25.0 


Pre-electronic mediums 


Paper 


58 


33.0 


92 


52.3 


9 


5.1 


15 


8.5 


Microform 


6 


26.1 


14 


60.9 


3 


13.0 


0 


0.0 


Other 


2 


16.7 


6 


50.0 


2 


16.7 


2 


16.7 


Optical mediums 


CD-ROM 


9 


12.9 


41 


58.6 


4 


5.7 


15 


21.4 


WORM (write once, read many disk) 


0 


0.0 


0 


0.0 


0 


0.0 


1 


100.0 


DVD (digital video disk) 


0 


0.0 


0 


0.0 


0 


0.0 


1 


100.0 


Other 


0 


0.0 


1 


100.0 


0 


0.0 


0 


0.0 


Electronic mediums 


Magnetic tape 


3 


16.7 


12 


66.7 


1 


5.6 


2 


11.1 


Floppy diskette 


3 


7.1 


29 


69.0 


5 


11.9 


5 


11.9 


Hard drive 


2 


6.7 


24 


80.0 


0 


0.0 


4 


13.3 


Other 


1 


10.0 


5 


50.0 


4 


40.0 


0 


0.0 



NOTE: Percents may not add to 100 because of rounding. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product Assessment Questionnaire, 1998. 



Format Types Used 

Databases. Responses to all formats used are shown in table 4a. Wide 
Area Information Server (WAIS) is the most common type of database 
identified (22 responses), followed by Oracle (17 responses), and dBase 
(9 responses). In some cases, WAIS is reported because the products 
surveyed are made available through GPO Access. The 44 responses in the 
“other” category reveal few multiple responses except for Microsoft Access 
that received 5 “write-in” responses in this category. 

WAIS (24 percent) and Oracle (14 percent) are the primary types of 
databases used (table 4a). Ninety-one percent of the respondents who 
checked WAIS as one of the databases used also indicated that the use of 
WAIS is a common agency practice, while only one respondent indicated 
that WAIS is agency mandated (table 4b). However, only 44 percent of the 
respondents identified the use of Oracle as a common agency practice. 
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but 39 percent of respondents indicated their use of Oracle is agency 
mandated. 

Spreadsheets. For spreadsheet formats used, Excel and Lotus 1-2-3 
received 33 and 23 responses, respectively (table 4a). When respondents 
were asked to choose one of the databases as the primary type used, 
59 percent chose Excel, while only 33 percent chose Lotus 1-2-3. Close to 
7 1 percent of the respondents also identified the use of Excel as a common 
agency practice as compared to 38 percent who indicated the use of Lotus 
1-2-3 as a common agency practice (table 4b). 

Tagged mark-up. Hypertext Markup Language (HTML) is both the most 
commonly used tagged markup language (157 responses) andi\\Q primary 
type of tagged markup language used (89 percent; table 4a). The 
Government agencies surveyed seldom use Extensive Markup Language 
(XML) (2 responses), and Standard Generalized Markup Language 
(SGML) (14 responses). This is noteworthy since SGML is one of the few 
formats that NARA accepts for electronic records. 

Even given the fact that HTML is the primary type of tagged markup 
format used, 72 percent of the respondents reported that HTML is used as a 
common agency practice, while only 13 percent reported that its use is 
mandated by the agency (table 4b). Sixty percent of the respondents who 
use SGML for their online products reported it as a common agency 
practice, while only 13 percent reported that its use is mandated by the 
agency. 

Image formats. Portable Document Format (PDF) is the most common 
image format (132 responses) and the primary type of format used 
(49 percent) by the agencies surveyed in this study (table 4a). The use of 
PDF is followed by GIF (99 responses), JPEG (77 responses), then TIFF 
(36 responses) as image formats used. Perhaps PDF is the most commonly 
used format by the agencies surveyed because the Federal Government 
disseminates a wide range and large number of forms and documents that 
must be printed in the exact format in which they are created. 

Almost 64 percent of respondents reported that PDF is a common agency 
practice, while 16 percent reported it is mandated by the agency 
(table 4b). While a higher percentage of respondents reported using GIF 
(69 percent) and JPEG (71 percent) as a common agency practice, PDF is 
the most used agency-mandated image format (16 percent). 

Audio formats. The number of responses reported in this category reflects 
the small numbers of products surveyed that contain sound (see table 2a). 
WAV (12 responses) is the most commonly used sound format followed by 
AU (5 responses), and AIFF with 1 response (table 4a). WAV is also the 
primary type of audio format used (73 percent). 
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Table 4a. 



Frequency and percent of formats used, 
by the type of format used and primary 
type of format used 



Format 


Type of format used* 


Primary type of 
format used^ 




Number 


Percent 


Number 


Percent 



Database 


WAIS 


... 22 


9.1 


22 


24.4 


Oracle 


... 17 


7.0 


13 


14.4 


dBase 


9 


3.7 


8 


8.9 


Sybase 


4 


1.7 


1 


1.1 


MARC 


2 


0.8 


3 


3.3 


Other 


... 44 


18.2 


40 


44.4 


Spreadsheet 


Excel 


... 33 


13.6 


30 


58.8 


Lotus 1-2*3 


... 23 


9.5 


17 


33.3 


Other 


3 


1.2 


2 


3.9 


Tagged markup 


HTML 


.. 157 


64.9 


152 


88.9 


SGML 


.. 14 


5.8 


8 


4.7 


XML 


2 


0.8 


0 


0.0 


Other 


12 


5.0 


10 


5.8 


Image 


PDF 


.. 132 


54.5 


84 


48.8 


GIF 


.. 99 


41.1 


43 


25.0 


JPEG 


77 


31.8 


16 


9.3 


TIFF 


36 


14.9 


4 


2.3 


Other 


19 


7.9 


13 


7.6 



Table 4a. 

Frequency and percent of formats used, 
by the type of format used and primary 



type of format used (continued^ 




Format 


Type of format used' 


Primary type of 
format used^ 


Number 


Percent 


Number 


Percent 



Audio 



WAV 


12 


5.0 


11 


73.3 


AU 


5 


2.1 


0 


0.0 


AIFF 


1 


0.4 


0 


0.0 


Other 


4 


1.7 


3 


20.0 



Video 



MPEG 


9 


3.7 


4 


26.7 


MOV 


7 


2.9 


5 


33.3 


AVI 


4 


1.7 


4 


26.7 


Other 


1 


0.4 


1 


6.7 


Text 


ASCII 


. 122 


50.4 


106 


80.9 


ANSI 


11 


4.5 


0 


0.0 


RTF 


9 


3.7 


5 


3.8 


Other 


17 


7.0 


17 


13.0 


Word processing 


Word Perfect 


75 


31.0 


62 


63.9 


Microsoft Word .... 


55 


22.7 


21 


21.6 


Other 


23 


9.5 


10 


10.3 


Other 


5 


2.1 


3 


60.0 



'Column percents do not add to 100 because respondents could 
choose more than one item. 

^Percents do not add to 100 due to nonresponse. 

SOURCE: National Commission on Libraries and Information 

Science, Government Information Product Assessment 
Questionnaire, 1998. 



Sixty-two percent or eight of the agency respondents who indicated using 
WAV reported it as a common agency practice; only two respondents 
(15 percent) reported that WAV is an agency-mandated standard 
(table 4b). Perhaps it is not surprising that WAV is the most commonly 
used audio format; since it was built into Windows95, it has become the de 
facto standard for sound on PCs. AIFF is the standard audio format for 
Macintosh computers (PC Webopaedia; see www.pcwebopaedia.com). 
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Table 4b. 

Number and percent of formats used, by the standard for each format used 



Standard for each format used 



Format 


Agency mandated 


Common agency 
practice 


Other 


None 




Number 


Percent 


Number 


Percent 


Number 


Percent 


Number 


Percent 



Database 

Oracle 7 

WAIS 1 

MARC 1 

Sybase 0 

dBase 0 

Other 2 

Spreadsheet 

Lotus 1-2-3 6 

Excel 4 

Other 0 

Tagged markup 

HTML 21 

SGML 2 

XML 0 

Other 0 

Image 

PDF 21 

GIF 11 

JPEG 7 

TIFF 2 

Other 3 



38.9 


8 


44.4 


1 


5.6 


1 


5.6 


4.3 


21 


91.3 


0 


0.0 


0 


0.0 


33.3 


1 


33.3 


0 


0.0 


0 


0.0 


0.0 


4 


80.0 


0 


0.0 


0 


0.0 


0.0 


8 


80.0 


0 


0.0 


0 


0.0 


4.4 


21 


46.7 


12 


26.7 


9 


20.0 



25.0 


9 


37.5 


3 


12.5 


5 


20.8 


11.8 


24 


70.6 


0 


0.0 


4 


11.8 


0.0 


0 


0.0 


0 


0.0 


2 


50.0 



13.3 


114 


72.2 


6 


3.8 


15 


9.5 


13.3 


9 


60.0 


2 


13.3 


1 


6.7 


0.0 


1 


33.3 


0 


0.0 


1 


33.3 


0.0 


7 


53.8 


1 


7.7 


4 


30.8 



15.9 


84 


63.6 


15 


11.4 


11 


8.3 


11.0 


69 


69.0 


11 


11.0 


8 


8.0 


9.1 


55 


71.4 


10 


13.0 


4 


5.2 


5.4 


22 


59.5 


10 


27.0 


2 


5.4 


13.6 


6 


27.3 


10 


45.5 


2 


9.1 



Video formats. As with the audio formats used, the even smaller number 
of responses reported in this category also reflect the small numbers of 
products surveyed that contain moving images. Table 4a shows that 
Moving Picture Experts Group (MPEG) (9 responses) is the most 
commonly used format, followed by MOV (7 responses) and Audio Video 
Interleave (AVI) (4 responses). MPEG may be more commonly used since 
it generally produces better quality video than AVI (PC Webopaedia). Of 
all the video formats used, however, the primary type of video format used 
is MOV (33 percent), followed by MPEG and AVI 
(27 percent each). 
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Table 4b. 

Number and percent of formats used, by the standard for each format used (continued) 


Format 


Standard for each format used 


Agency mandated 


Common agency 
practice 


Other 


None 


Number 


Percent 


Number 


Percent 


Number 


Percent 


Number 


Percent 


Audio 


















WAV 


2 


15.4 


8 


61.5 


0 


0.0 


2 


15.4 


AU 


4 


66.7 


0 


0.0 


0 


0.0 


1 


16.7 


AIFF 


1 


50.0 


0 


0.0 


0 


0.0 


0 


0.0 


Other 


0 


0.0 


1 


20.0 


2 


40.0 


1 


20.0 


Video 


















MOV 


0 


0.0 


5 


62.5 


0 


0.0 


2 


25.0 


MPEG 


1 


10.0 


5 


50.0 


1 


10.0 


2 


20.0 


AVI 


1 


20.0 


3 


60.0 


0 


0.0 


0 


0.0 


Other 


0 


0.0 


0 


0.0 


1 


50.0 


0 


0.0 


Text 


















ASCII 


14 


11.4 


87 


70.7 


6 


4.9 


14 


11.4 


RTF 


4 


40.0 


4 


40.0 


1 


10.0 


0 


0.0 


ANSI 


1 


8.3 


10 


83.3 


0 


0.0 


0 


0.0 


Other 


3 


16.7 


11 


61.1 


2 


11.1 


1 


5.6 


Word processing 


















Word Perfect 


19 


25.3 


45 


60.0 


1 


1.3 


8 


10.7 


Microsoft Word 


8 


14.3 


33 


58.9 


12 


21.4 


2 


3.6 


Other 


1 


4.2 


6 


25.0 


12 


50.0 


3 


12.5 


Other 


2 


40.0 


1 


20.0 


1 


20.0 


0 


0.0 



NOTE: Percents do not add to 100 due to nonresponse. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product Assessment Questionnaire, 1998. 



Of the respondents who reported using MPEG, 50 percent indicated its use 
is a common agency practice, while only 1 respondent (10 percent) reported 
that its use is agency mandated. Sixty-three percent of the respondents 
reported that MOV is used as a common agency practice, and none 
indicated that its use is agency mandated (table 4b). 

Text formats. ASCII is by far the most commonly used text format 
(122 responses) and the primary type of text format used (81 percent; table 
4a). The second most commonly used text format is ANSI (1 1 responses) 
followed by Rich Text Format (RTF) (9 responses). Seventy-one percent 
(87) of the respondents reported that their use of ASCII is a common 
agency practice, as compared to 1 1 percent ( 1 4) who reported its use is 
agency mandated (table 4b). 
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Word processing formats. Between the two most popular word- 
processing software packages, Microsoft Word and WordPerfect, the latter 
(75 responses) is more commonly used than Microsoft Word 
(55 responses; table 4a). These responses are also consistent with the 
primary type of word processing used. Sixty-four percent of respondents 
reported WordPerfect as the primary type of format used while only 22 
percent of respondents reported Microsoft Word as the primary type of 
format used. PageMaker received the largest number of responses (5) in the 
“other” category. Nineteen respondents (25 percent) reported that 
WordPerfect is an agency-mandated format standard, while only 8 
respondents (14 percent) indicated that Microsoft Word is an agency- 
mandated format standard (table 4b). 

Summary of format types used. Each of the 242 respondents from the 24 
agencies surveyed was asked to identify the primary type of format used of 
each of the categories. The primary types of formats used in each category 
are WAIS, Excel, HTML, PDF, ASCII, and to a lesser degree, WAV and 
MOV. 



User Interfaces 

Online approaches. Question 9 on the survey refers to online approaches 
used. Eighty-five percent of the respondents reported that their product is in 
an online medium (table 5). These respondents were then asked to respond 
to a set of questions on user interfaces supported and web design 
approaches. 



Tables. 

Number and percent of products reported as 


being in an online medium 


Product characteristic 


Number Percent 


Is this product in an online medium? 

Yes 


206 85.1 


No 


33 13.6 


NOTE: N = 242; 3 cases were not ascertained. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product 


Assessment Questionnaire, 1998. 





User interface supported. Table 6a shows that Netscape Navigator 
(195 responses) is a more commonly supported browser than Internet 
Explorer (170 responses). However, close to 70 percent of agency 
respondents indicated that both of these browsers are almost equally 
supported as a common agency practice rather than an agency-mandated 
standard (table 6b). 
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Table 6a. 

Number and percent of online approaches 



used, by type of online tool 


used 




Online approach 


Type of online 
tool used 


Number 


Percent 


User interfaces supported 






Netscape 


195 


93.3 


Internet Explorer 


170 


81.7 


FTP 


40 


19.1 


Telnet 


27 


12.9 


Nongraphical/dial-up shell 


15 


7.2 


Other 


22 


10.5 



Web design approaches 

Basic HTML only 

Tables 

CGI Scripts 

Frames 

Use of Javascript 

Use of Java Applets 

XML 

Other 

Bulletin board systems (BBS) 

Graphical interface/browser 7 3.4 

NOTE: Percents do not add to 100 because respondents could 
choose more than one item. 

SOURCE: National Commission on Libraries and Information 
Science, Government Information Product Assessment 
Questionnaire, 1998. 



150 71.8 

111 53.1 

66 31.6 

53 25.4 

43 20.6 

23 11.0 

11 5.3 

25 12.0 



In addition, respondents reported that file transfer protocol 
(FTP), Telnet, and nongraphical/dial-up shells are also 
supported by their agencies (table 6a). Designs that 
support LYNX, a text-based browser, account for 12 of the 
22 responses in the ‘"other” category. The number of 
responses for the category nongraphical/dial-up shell are 
low (15 responses), especially given the need for agencies 
to comply with the American with Disabilities Act by 
making their sites more accessible to the visually and 
hearing impaired. 

Like the browsers, the other user interfaces supported are 
primarily supported as a common agency practice rather 
than an agency-mandated standard. Almost 83 percent of 
the 40 respondents who reported their agency supports 
FTP also reported it is a common agency practice, while 79 
percent of the 27 respondents who reported supporting 
Telnet also indicated it as a common agency practice (table 
6b). No respondents reported that Telnet is an agency- 
mandated standard; however, 8 percent reported that FTP 
is an agency-mandated standard for their surveyed 
products. Seventy-five percent of the respondents reported 
the support of a nongraphical/dial-up shell as a common 
agency practice while only 13 percent indicated that it is 
agency mandated. 

Web design approaches. Various web design approaches 
used, in descending order, are HTML 
(150 responses), tables (111 responses), CGI Scripts 
(66 responses), frames (53 responses). Javascript 
(43 responses), Java Applets (23 responses), and XML (1 1 
responses; table 6a). ColdFusion was reported in three of 
the responses in the “other” category. 

The use of these web design approaches is overwhelmingly 
a common agency practice rather than an agency-mandated 
standard (table 6b). Basic HTML — tags that consistently 
display content in a similar fashion by the most popular 
browsers — is the only approach to which almost one-fifth 
(18 percent) of the respondents reported that its use is 
agency mandated. Less than 

10 percent of the respondents using each of the other 
approaches indicated that they are agency-mandated 
standards. Since the use of frames. Javascript, Java 
Applets, and XML may not be supported or enabled for 
many users’ browsers, the agencies surveyed appear to be 
adopting them slowly, if at all. 
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Table 6b« 

Number and percent of online approaches used, by the standard for each online tool used 



Standard for each online tool used 



Online approach 


Agency mandated 


Common agency 
practice 


Other 


None 




Number 


Percent 


Number 


Percent 


Number 


Percent 


Number 


Percent 



User interfaces supported 



Netscape 


36 


18.3 


139 


70.6 


4 


2.0 


17 


8.6 


Internet Explorer 


25 


14.5 


120 


69.4 


4 


2.3 


22 


12.7 


FTP 


3 


7.5 


33 


82.5 


0 


0.0 


3 


7.5 


Nongraphical/dial-up shell 


2 


12.5 


12 


75.0 


0 


0.0 


1 


6.3 


Telnet 


0 


0.0 


22 


78.6 


1 


3.6 


4 


14.3 


Other 


1 


4.8 


18 


85.7 


0 


0.0 


1 


4.8 


Web design approaches 


Basic HTML only 


27 


17.9 


105 


69.5 


1 


0.7 


16 


10.6 


Tables 


10 


8.9 


88 


78.6 


3 


2.7 


9 


8.0 


CGI Scripts 


4 


6.0 


55 


82.1 


0 


0.0 


7 


10.4 


Use of Javascript 


3 


6.8 


30 


68.2 


6 


13.6 


4 


9.1 


Frames 


2 


3.7 


44 


81.5 


3 


5.6 


4 


7.4 


Use of Java Applets 


2 


8.3 


15 


62.5 


3 


12.5 


3 


12.5 


XML 


0 


0.0 


10 


83.3 


1 


8.3 


0 


0.0 


Other 


3 


11.1 


17 


63.0 


2 


7.4 


4 


14.8 


Bulletin board systems (BBS) 


Graphical interface/browser 


1 


12.5 


5 


62.5 


0 


0.0 


1 


12.5 



NOTE: Percents do not add to 100 due to nonresponse. 

SOURCE; National Commission on Libraries and Information Science, Government Information Product Assessment Questionnaire, 1998. 



Searchability of Product 

Searchability of an electronic product is important for users because it 
allows them to effectively access the information they need. Most electronic 
products are searchable either by full-text with no fielding 
(74 responses) and/or by full-text and field (99 responses; table 7). The 
“view only” category contains a higher number of responses than expected 
(79 responses). 

The “other” category contains the following common responses: 

• Inapplicable because product is in a paper medium (most common 
response); 

• In PDF, which is not searchable; and 

• Product is indexed by field only. 



ERIC 



30 



48 



Assessment of Electronic Government Information Products 



Table 7. 

Number and percent of responses regarding 
searchability of the product 






Product characteristic 


Number 


Percent 


Indexed by full-text and field 


99 


40.9 


Available as “view only” — non-searchable 


79 


32.6 


Included as part of a full-text searchable database with no 
fielding 


74 


30.6 


Other 


32 


13.2 



NOTE: Percents do not add to 100 because respondents could choose more than one item. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire, 1998. 



Product “Host” 

Most of the products surveyed (199 responses) were hosted by the agency 
that created them, although other agencies or institutions might also host the 
products since respondents were asked to “check all that apply” for this 
question (table 8). There are fewer responses for products hosted by another 
agency (42 responses), a contractor (17 responses), and an educational 
institution (9 responses). 



Table 8. 

Number and percent of responses regarding 
where the product is hosted 






Product host 


Number 


Percent 



Your agency 

AiiOthcr agency 

Contractor 

Educational institution 
Other 



199 


82.2 


42 


17.4 


17 


7.0 


9 


3.7 


15 


6.2 



NOTE: Percents do not add to 1 00 because respondents could choose more than one item. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire, 1998. 
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Retrievability of Product 

In order to ensure broad access to the product, the public should be able to 
download and save electronic Government information products without 
restrictions (GPO, 1996, p. 7). Responses to Question 1 1 indicate that for 
the most part, products surveyed for this study can be downloaded and 
saved without restrictions (173 responses; table 9). Responses in the 
second category indicate that some products cannot be downloaded or saved 
(20 responses). A small number of products (14) cannot be downloaded or 
saved because their use requires proprietary software that is not freely 
distributed (table 9). Common write-in responses in the “other” category 
include: 

• Can be downloaded and saved, but subject to restrictions. 

• Can be printed from browser, but not downloaded. 

• Product available only in paper. 

The United States Advisory Council on the National Information 
Infrastructure, in its publication “A Nation of Opportunity,” identifies as 
one of the basic principles of Government information and services that “the 
Federal Government should not charge for making its information 
available... nor charge for access to that information” (GPO, 1996, p. 28). 



Table 9. 

Number and percent of responses concerning 
the retrievability status of the product 






Product and software characteristic 


Number 


Percent 


Can be downloaded, saved, and is not subject to any 

restrictions on use or re-use by the end user 


173 


71.5 


Cannot be downloaded, saved, and/or re-used because it is 
part of a database and does not exist as a distinct 
product 


20 


8,3 


Cannot be downloaded, saved, and/or re-used because it 
requires proprietary software that is not freely 
distributable 


14 


5.8 


Other 


41 


16.9 


NOTE: Percents do not add to 100 because respondents could choose more than one item. 
SOURCE: National Commission on Libraries and Information Science, 

Government Information Product Assessment Questionnaire, 1 998. 
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Section C Responses 
(Planned Product Profile) 



Section C contains a series of questions related to the future product profile. 
Respondents were asked questions about changes in the types of data, 
mediums, and formats used and reported on in Section B of the 
questionnaire. Respondents also were asked to identify the time span in 
which the changes would occur and to describe the planned changes. 



Types of Data 

The first question in this section of the questionnaire asked respondents 
about plans to discontinue publication of the product. Only 5 percent (12) 
of the respondents planned to discontinue the product (table 10). Several of 
the most commonly listed responses provided for discontinuation of a 
product was that the product was a one-time “prototype” or that the paper 
version of the product would be discontinued. 



Table 10. 

Number and percent of respondents reporting plans to 
discontinue publication of the product 




Product characteristic 


Number 


Percent 


Are there any plans to discontinue 
publication of this product? 

Yes 


12 


5.0 


No 


230 


95.0 


SOURCE: National Commission on Libraries and Information Science, Government Information Product 


Assessment Questionnaire, 1998. 







Table 11 shows responses to question 13 about the kinds of data 
(i.e., bibliographic, textual, graphical) the product will contain in the future. 
The majority of respondents (76 percent) reported that the agency plans no 
changes to the product. Twenty-one percent reported that the agency would 
add one or more new types of data. A total of 3 percent reported either the 
discontinuation of one type of data (0.4 percent), or a complete change to 
new data types (2.6 percent). Several respondents reported that the changes 
in data types would include adding audio or video and multimedia. 
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Table 11, 

Number and percent of responses regarding the planned changes to the 
type of data contained in the product 



Number 


Percent 


178 


76.4 


48 


20.6 


1 


0.4 


6 


2.6 



Product characteristic 



What kind of data will this product contain? 

Retain existing type(s) of data, no changes planned 

Retain existing type(s) of data and add items of one or 

more new types of data 

Discontinue one or more types 

Change to new type(s) of data 



NOTE: N = 242; 9 cases are missing. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire, 1998. 



Most agency respondents reported that these changes in data types would 
mainly occur in the short term (40 responses) and, to a lesser degree, in the 
medium term (24 responses; table 12). Most respondents skipped the 
question about long-term plans for changing data types. Respondents noted 
in the “please specify” categories in questions 13c and 13e indicate that 
respondents’ plans for product changes have not yet been solidified. 



Table 12. 

Number and percent of responses regarding the timeframe for planned 
changes to the type of data contained in the product 



Product characteristic 



Number 



Percent 



Change(s) will occur in the: 

Shortterm; within 1 year or less 40 72.7 

Medium term: within 2 to 5 years 24 43.6 



NOTE: Percents do not add to 100 because respondents could choose more than one item. Numbers 
reflect respondents who indicated changes. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire, 1998. 



Types of Mediums 

Responses to changes in types of mediums parallel those for changes in data 
types. Seventy-six percent of the respondents reported no plans to change 
mediums (table 13). Eighteen percent of respondents reported that they are 
planning to add one or more mediums, 2 percent indicated they will 
discontinue one or more mediums, and 3 percent reported they will change 
to a new type of medium. 
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Table 13. 

Number and percent of responses regarding the planned 
changes to the mediums used for the future 




Product characteristic 


Number 


Percent 


What kind of medium(s) will this product use? If 
product is delivered in more than one medium, respond 
for all mediums. 

Retain existing medium(s), no changes planned 


178 


76.1 


Retain existing medium(s) and add items of one or 
more new types of mi 


42 


17.9 


Discontinue one or more types 


4 


1.7 


Change to new type(s) of medium 


8 


3.4 


No agency mandated medium applies 


0 


0.0 



NOTE: N = 242; 8 cases are missing; 2 cases were not ascertained. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire, 1998. 



The two most frequently mentioned additions to medium types are to 
provide web access to the product, and to make the product available on 
CD-ROM. Most of the respondents (35) who reported changes in medium 
types indicated that the changes will occur in the medium term; 
21 respondents indicated that these changes will occur in the short term 
(table 14). Again, most respondents skipped the question about long-term 
plans for changing product mediums. The few respondents who provided 
descriptions of their long-term plans mentioned that they will produce the 
product in multiple mediums (paper and web), or that paper items will be 
migrated to the web. Other respondents indicated that their long-term plans 
are undetermined or undefined. 



Table 14. 

Number and percent of responses regarding the timeframe for planned 
changes to product medium used 


Product characteristic 


Number 


Percent 


Change(s) will occur in the: 






Short term: within 1 year or less 


21 


37.5 


Medium term: within 2 to 5 years 


35 


62.5 



NOTE; Numbers reflect respondents who indicated changes. 

SOURCE; National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire, 1998. 
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Types of Formats 

One might expect to see more dramatic changes in types of formats since 
the range of formats is varied and broad (i.e., database, spreadsheet, tagged 
markup, image, etc.). The pattern of responses to question 15 mirrors the 
responses to changes in types of data and mediums, except for the change to 
new types. Seventy-two percent of the respondents reported no changes in 
format types. Eighteen percent indicated that they are planning to add one 
or more formats, while 9 percent reported they will change to new format 
types (table 15). This change to new format types is the largest percentage 
change in this category as compared to changes to new types of data (3 
percent; table 11) and new types of mediums (also 
3 percent; table 13). Respondents who provided specifics about the 
changes to new format types indicated these new types would be PDF and 
XML. 



Table 15. 

Number and percent of responses regarding the planned 
changes to the formats the product will contain 



Product characteristic 



Number 


Percent 


167 


71.7 


42 


18.0 


21 


9.0 


1 


0.4 


1 


0.4 



What kind of format(s) will this product contain? 

Same as existing format(s), no changes planned 

Retain existing format(s) and add one or more new format 

types 

Change to new format types 

Discontinue one or more types 

No agency-mandated format applies 

NOTE: N = 242; 9 cases are missing; 1 case was not ascertained. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire, 1998. 



The majority of respondents who reported changes indicated that they will 
occur in the short term (36 responses), and/or the medium term 
(32 responses; table 16). The majority of respondents did not answer the 
question about long-term plans for changing formats. 



Table 16. 

Number and percent of responses regarding the timeframe 
for planned changes to the product format used 



Product characteristic 



Number 



Percent 



Change(s) will occur in the: 

Short term: within 1 year or less 36 55.4 

Medium term: within 2 to 5 years 32 49.2 



NOTE: Total frequency reflects respondents reporting changes only. Percents do not add to 100 because 
respondents could choose more than one item. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire, 1998. 
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Section D Responses 
(Other Information) 



Section D of the questionnaire contains a variety of questions in an effort to 
answer some of the critical issues of public access to electronic information 
products: 

• Metadata, 

• Permanent public access (i.e., provided by what agency and how), 

• Permanent retention, 

• Ensuring authenticity, 

• Updating/upgrading plans, 

• User fees, 

• Licensing, and 

• Public domain. 



Metadata 

Metadata, data about data, are important for public access. Metadata 
refers to describing the content of a document or record allowing users to 
find Government information more effectively. Examples of metadata 
include Government Information Locator Service (GILS) and machine- 
readable cataloging (MARC) records. To that end, one of the requirements 
of the Government Printing Office Electronic Information Access 
Enhancement Act of 1993 (Public Law 103-40) was that the Superintendent 
of Documents maintain an electronic directory of Federal electronic 
information (44 U.S.C., Section 4101). 

Only 27 percent of agency respondents reported that their products have a 
metadata record, while 69 percent reported no metadata record exists for 
their products (table 17). In the followup question, most respondents 
identified their metadata records as either MAJIC or GILS. Another 
5 percent indicated they do not know if a metadata record exists. 



Table 17. 

Number and percent of respondents reporting 
a metadata record for the product 






Product characteristic 


Number 


Percent 


Is there a metadata record for this product 
(e.g., GILS, MARC)? 

Yes 


65 


26.9 


No 


166 


68.6 


Don’t know 


11 


4.5 



SOURCE: National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire, 1998. 
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Permanent Public Access 

In an electronic age, permanent public access to Government information, a 
critical concept in information resources management, presents far-reaching 
challenges to the Federal Depository Library Program, Congress, Federal 
agencies, and ultimately the American public. GPO indicates that 
permanent public access ''means that electronic Government information 
products within the scope of the FDLP remain available for continuous, no- 
fee public access through the program” (GPO, 1998, p. 19). GPO 
recognizes and acknowledges its responsibility to provide ongoing public 
access to the electronic Government information available through the 
FDLP. However, in a decentralized networked environment, agencies are 
asked to share the responsibility for building, storing, disseminating, and 
preserving a broad range of electronic information products in order to 
ensure continued public access. 

Agency respondents reported that permanent public access is primarily 
provided by their agency (177 responses), by another agency 
(51 responses), and/or by some other entity (20 responses; table 18). 
Respondents reported that permanent public access is not provided for 
28 products (table 18). However, on closer examination, the responses to 
the "please specify” questions indicate that either respondents may have 
misunderstood the concept of permanent public access (as opposed to 
current access), or they assumed other entities have this responsibility. 
Some of the common responses to "other” agencies include the Government 
Printing Office, National Archives and Records Administration (NARA), 
and contractors and vendors. These responses illustrate respondents’ lack 
of understanding about the difference between permanent public access to 
electronic information products through their own agencies or through 
partnerships with GPO, and permanent retention of official Government 
records through NARA. Furthermore, only 4 of the 28 products for which 
no permanent public access currently is provided have future plans for 
providing permanent public access (table 19). 
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Table 18. 

Number and percent of responses regarding tbe 
entity providing permanent access to tbe product 


Product characteristic 


Number 


Percent 


Permanent public access to this 






product is currently provided by: 






Your agency 


177 


73.1 


Another agency 


51 


21.1 


Other 


20 


8.3 


No permanent public access provided 


28 


58.3 



NOTE: Percents do not add to 100 because respondents could choose more than one item. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire, 1998. 



Table 19. 



Number and percent of responses regarding products 
for which access will be provided in the future 



Product characteristic 


Number 


Percent 


Are there plans to provide permanent public access in 
the future for this product? 

Yes 


4 


12.9 


No 


24 


80.6 



NOTE: Only those respondents (N=28) who indicated there is currently no permanent public access 
provided for this product were asked to complete this question. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire, 1998. 



Permanent Retention 

The mission of the National Archives and Records Administration is 
distinct from that of GPO. NARA’s mission is to preserve and provide 
public access to permanently valuable records of the Federal Government. 
Federal agencies are responsible for transferring products to NARA that are 
scheduled as permanent records (i.e., official records of the Federal 
Government as defined by the Federal Records Act). Under 36 CFR 
1228.188, mediums approved for transfer include open reel magnetic tape, 
magnetic tape cartridge, and CD-ROM. Agencies currently may not 
transfer to NARA electronic records that are in a format dependent on 
specific hardware and software. However, SGML tags are permitted on 
electronic textual documents as are records written in ASCII or Extended 
Binary Coded Decimal Interchange Code (EBCDIC) with all control 
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characters and other non-data characters removed (Lewis Bellardo, Deputy 
Archivist of the U.S. in a written response to agency questions, October 14, 
1998). 

The responses to the questions on permanent retention may reflect the 
current status of transferring permanent electronic records to NARA (see 
questions findings). Only 34 percent of agency respondents reported that 
their products are scheduled for permanent retention by NARA (table 20). 
Sixty- four percent reported their products are not scheduled for retention, 
while another 3 percent reported they do not know if the product is 
scheduled for retention. However, it should be pointed out that at the time 
of the survey, the schedule that would have covered electronic records of 
permanent value was unenforceable under a court case declaring it null and 
void; therefore, these figures may be unreliable. 



Table 20. 

Number and percent of responses regarding 
permanent retention of the product 






Product characteristic 


Number 


Percent 


Is this product scheduled for permanent retention by the 
National Archives and Records Administration? 

Yes 


82 


33.9 


No 


154 


63.6 


Don’t know 


6 


2.5 



SOURCE: National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire, 1998. 



Ensuring Authenticity 

Although 64 percent of respondents reported that their agency ensures 
authenticity for the products surveyed (table 21), responses to the open- 
ended question about how the agency attests to authenticity indicate that 
respondents may not fully understand the concept. 

Authentication refers to the process agencies use to ensure the public that 
the product is an official legitimate product created and produced by the 
Federal Government agency and no other source (see glossary, p. E-17). 
Ensuring authentication includes technical as well as policy considerations. 
Some technical examples of authentication include digital signature 
technology, special watermarks, disclaimers, or statements on the products. 
Respondents provided answers that address how the agency ensures that 
information or data in the product are valid or reliable — an 
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important process, but not the same concept as authenticity. Common 
responses include the following: 

• Program office verifies data. 

• Review CD-ROM contents before public release. 

• Regulations and source/reliability statement regarding data sources. 

• Review and approval within agency. 

• Source of content is the same as the hardcopy version. 

• Test reliability of data every 5 years, or more often. 

• Publications are subjected to review by subject matter expert and peer 
review. 



Table 21. 

Number and percent of respondents wbo reported 
the agency ensures authenticity for the product 




Product characteristic 


Frequency 


Percent 


Does the agency ensure authenticity 
(official status determination) for this product? 

Yes 


154 


63.6 


No 


86 


35.5 


Don’t know 


2 


0.8 


NOTE: Percents may not add to 100 because of rounding. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product 


Assessment Questionnaire, 1998. 







Updating/Refreshing Plans 

Twenty percent of respondents reported that their products are updated 
annually, followed by daily (16 percent), monthly (12 percent), and weekly 
(5 percent; table 22). However, the majority (47 percent) of respondents 
checked the ‘‘other” response category. The write-in responses covered a 
broad range of time periods in which products are updated. Below is a 
sampling of multiple responses: 

• Quarterly, 

• As needed, 

• Irregularly, 
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• Not updated, 

• Semi-annually, 

• Every 2 years, 

• Periodically, and 

• Twice a month with old version staying on line. 



Table 22. 

Number and percent of responses regarding 
how frequently the product is updated or refreshed 



Product characteristic 



Number 



Percent 



How frequently is this product updated or refreshed? 



Annually 48 19.8 

Daily 39 16.1 

Monthly 30 12.4 

Weekly 12 5.0 

Other 113 46.7 



SOURCE: National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire, 1998. 



Changing Supporting Technology 

The majority of the respondents (71 percent) reported that there are no 
plans to change the product’s supporting technology (table 23). Twenty- 
eight percent of respondents reported plans to change the product’s 
supporting technology. 



Table 23. 



Number and percent of responses regarding 
the plans for supporting technology of the product 



Product characteristic 


Number 


Percent 


Are there plans for changing the product’s supporting 
technology? 

Yes 


68 


28.1 


No 


171 


70.7 


Don’t know 


2 


0.8 



NOTE: N = 242; 1 case not ascertained. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire, 1998. 
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User Fees 

Public access to no-fee Government information products is one of the core 
principles upon which the FDLP is based. However, users might be 
charged a fee if they order certain types of electronic Government 
information products directly from GPO or the agency that created the 
product. Nine percent of respondents reported that all users are charged 
fees, while 20 percent reported some users are charged fees. The majority 
(72 percent) of agency respondents reported that there are no fees charged 
to access or use the product surveyed (table 24). The followup question 
asks about specific fee amounts and the reasons for the charge. The 
responses to this question vary greatly. A few common responses include 
the following: 

• No charge for web access. 

• Single paper copy free; charge for additional copies. 

• No subscription fee to libraries and some constituencies. 

• Files can be downloaded from the Internet for free. There is a charge 
for published books. 

• Fees are for paper products only. 



Table 24. 



Number and percent of respondents reporting 
that user fees are charged for the product 



Product characteristic 


Number 


Percent 


Is a user fee charged for this product? 






Yes, for some users 


48 


19.8 


Yes, for all users 


21 


8.7 


No 




71.5 



SOURCE: National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire, 1 998. 



Licensing 

Many Government agencies purchase licenses from vendors for search and 
retrieval software to be used with the product to make the data or 
information more accessible to users. Agencies negotiate various 
agreements with vendors about who can use the software free of charge. 
The majority of respondents (69 percent) reported that they do not license 
commercial search and retrieval software (table 25). For the remaining 
31 percent of respondents who have licensed commercial software, the 
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license covers use by all the key constituencies including agency personnel 
(73 responses), public users (69 responses), agency’s primary target 
constituencies (65 responses). Federal depository libraries 
(59 responses), and/or all libraries (59 responses; table 26). 



Table 25. 



Number and percent of respondents reporting about the use of licensed 
commercial search and retrieval software for the product 



Product characteristic 


Number 


Percent 


Has the agency licensed commercial search and retrieval 






software for use with this product? 






Yes 


76 


31.4 


No 


166 


68.6 



SOURCE: National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire, 1998. 



Table 26. 

Number and percent of responses regarding coverage 
by the agency software license 



Number 


Percent 


73 


96.1 


65 


85.5 


59 


77.6 


59 


77.6 


69 


90.8 


10 


13.2 



Product characteristic 



Does the agency’s license cover use by: 

Agency personnel 

Agency’s primary target constituencies . 

Federal depository libraries 

All libraries 

Public users 

Other 



NOTE: Percents do not add to 100 because respondents could choose more than one item. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire, 1998. 



Public Domain 

Public domain, a critical component of public access, means that the 
information, product, or publication is not copyrighted and therefore can be 
reproduced by anyone without obtaining copyright permission. One of the 
goals of an electronic FDLP is to provide public access to any Government 
information product free of copyright or copyright-like restrictions (GPO, 
1996, p. 2). The majority of respondents, 86 percent, indicated that all 
parts of their surveyed products are in the public domain (table 27). 
Another 10 percent indicated that part of the product is in the public 
domain, while 5 percent reported that the product is not in the 
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public domain. The followup question that requests an explanation of the 
second response (i.e., part of product is in the public domain) uncovered 
these typical responses: 

• Copyrighted tables are not in the public domain. 

• There are some copyright-protected logos and trademarks. 

• Includes copyrighted material that would require approval for 
reproduction. 

Respondents offered a wide variety of explanations for products that are not 
in the public domain: 



• Retrieval software is proprietary and use is licensed. 

• Commercial vendors lease the database for distribution. 

• Songs and performances are protected by copyright. 

• Books are available only to eligible blind patrons of our program, by 
law. 



Table 27. 



Number and percent of respondents reporting 
the public domain status of the product 



Product characteristic 


Number 


Percent 


Is this product in the public domain? 






Yes, for the entire product 


206 


85,5 


Yes, for part of the product 


24 


10,0 


No 


11 


4,6 



NOTE: N = 242; 1 case missing. Percents may not add to 100 because of rounding. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product 
i\ssessment Questionnaire, I 77 S. 



Section £ Responses 



The final section E of the questionnaire contains one open-ended 
“comments” question. These responses are too broad and disparate to 
provide a detailed itemization. Most of the comments are explanations of 
issues covered in the survey. However, below are a few comments that 
cover issues not directly addressed in the survey. 

• Our mission, mandated by the Americans with Disabilities Act (ADA), 
is to satisfy all browser requirements (e.g., ASCII browsers like LYNX 
through the latest versions of Netscape and Internet Explorer). 
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Study Questions 



Preferred Medium and 
Format Standards 



• We produce printed documents and link to electronic documents 
maintained on the GPO’s server. 

• The product is not published in any electronic form. It is a collection of 
individual products that are individually published. 

• I am very new to this area (2 weeks) and received significant contractor 
assistance in completing this form. 

• In addition to four other web sites, we will soon web-enable our 
database with some encrypted modules. 

• The database is intended to be accessible to the largest audience 
possible via free or public domain software whenever possible. 

• This information is available in PDF format on our website to ensure 
the integrity of the data. Coding in HTML (particularly tables) could 
lead to mistakes with such a large amount of numeric data. 

No respondents commented on the survey questionnaire, the project in 

general, or the process of filling out the survey. 



This section will use findings from two or more survey questions to provide 
additional information on some of the key issues explored in the study. The 
responses to these questions relate specifically to the products surveyed. 

The following questions were chosen because they address one or more of 
the critical study areas: preferred medium and format standards, permanent 
public accessibility, permanent retention, user fees, commercial licensing of 
search and retrieval software, and authenticity. 



Study Question 1: What combinations of preferred medium standards are 
currently used by the respondents? 

The agencies surveyed are creating and using (in descending order): 

• Products both in paper format and on the web. 

• Products both in CD-ROM and the web. 

• Products both in paper and CD-ROM. 

Since most of the agencies surveyed create products in more than one 
medium, what combinations of preferred mediums are they using? 

Of the respondents who indicated that paper was a medium used and the 
respondents who reported that CD-ROM was a medium used, only 
19 percent reported that they are using both paper and CD-ROM products 
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(table 28). Table 29 shows that of the respondents who reported that they 
use CD-ROM, and those who reported that they use the web as a medium, 
21 percent use both CD-ROM and the web as mediums. However, of the 
respondents who reported using paper and the respondents who reported 
using the web as a medium, 64 percent use both paper and the web (table 
30). Therefore, the respondents surveyed are creating and using products 
both in paper format and on the web much more often than they are creating 
and using products in CD-ROM and the web. An even smaller percentage 
of products is being created in paper and in CD-ROM. This confirms the 
earlier finding that paper and the web are the preferred mediums used by the 
agencies surveyed, but provides additional information about the 
combinations of mediums used. 



Table 28. 



Number and percent crosstabulations of products 
in both paper and CD-ROM formats 



Paper 


CD-ROM 


Total 


Yes 


No 


Number 


Percent 


Number 


Percent 


Number 


Percent 


Yes 


45 


18.6 


132 


54.5 


177 


73.1 


No 


25 


10.3 


40 


16.5 


65 


26.8 


Total 


70 


28.9 


172 


71.0 


242 


99.9 



NOTE: Percents may not add to 100 because of rounding. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire, 1998. 



Table 29. 

Number and percent crosstabulations of products 



in both CD-RO] 


M and web formats 


CD-ROM 


Web 


Total 


Yes 


No 


Number Percent 


Number Percent 


Number Percent 



Yes 51 21.1 19 7.9 70 29.0 



Yes 51 21.1 19 7.9 70 29.0 

No 153 63.2 19 7.9 172 71.1 

Total 204 84,3 38 15,8 242 100.1 



NOTE: Percents may not add to 100 because of rounding. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire, 1998. 
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Table 30, 



Number and percent crosstabulations of products 



Paper 


Web 


Total 


Yes 


No 


Number 


Percent 


Number 


Percent 


Number 


Percent 


Yes 


. 155 


64.0 


22 


9,1 


177 


73,1 


No 


49 


20,2 


16 


6,6 


65 


26,8 


Total 


, 204 


84,2 


38 


15,7 


242 


99,9 



NOTE: Percents may not add to 100 because of rounding. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire, 1998. 



Study Question 2: What combinations of preferred format standards are 
used by the respondents? 



The respondents are slightly more likely to use HTML in combination 
with PDF than they are to use HTML together with GIF, However, 
they are almost as likely to use HTML, GIF, and ASCII together as 
they are to use HTML, PDF, and ASCII together. 

Of the respondents who reported using HTML as a tagged markup format, 
and those who reporting using PDF as an image format, 39 percent reported 
the use of both HTML and PDF (table 31). Of the respondents who 
checked HTML, and those who checked GIF as an image format, 
36 percent checked that they used HTML in combination with GIF (table 
32), slightly less than those who used HTML and PDF in combination. 
Since PDF is the preferred image format used by agencies (table 4), this is 
not an unexpected finding. 



Table 31, 



Number and percent crosstabulations of products 
in both HTML and PDF formats 



HTML 


PDF 


Total 


Yes 


No 


Number 


Percent 


Number 


Percent 


Number 


Percent 


Yes 


94 


38,8 


63 


26,0 


157 


64,8 


No 


38 


15,7 


46 


19,0 


84 


34,7 


Total 


, 132 


54,5 


109 


45,0 


241 


99.5 



NOTE; Percents may not add to 1 00 because of rounding, 

SOURCE: National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire, 1998, 




48 



66 



Assessment of Electronic Government Information Products 



Table 32. 



Number and percent crosstabulations of products 
in both HTML and GIF formats 



HTML 


GIF 


Total 


Yes 


No 


Number 


Percent 


Number 


Percent 


Number 


Percent 


Yes 


86 


35.7 


70 


29.0 


156 


64.7 


No 


13 


5.4 


71 


29.5 


84 


34.9 


Total 


99 


41.1 


141 


58.5 


240 


99.6 



NOTE: Percents may not add to 1 00 because of rounding. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire, 1998. 



However, when the formats are used in combinations of three, it appears 
that respondents are almost as likely to use HTML, GIF, and ASCII 
(21 percent) together as they are to use HTML, PDF, and ASCII 
(22 percent) together (tables 33 and 34). 



Table 33. 

Number and percent of products that use HTML 
with GIF and ASCH formats 



Format 


Number 


Percent 


Both GIF and ASCII 


50 


20.7 


GIF only 


36 


14.9 


ASCII only 


27 


11.2 



SOURCE: National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire, 1998. 



Table 34. 

Number and percent of products that use HTML 
with PDF and ASCH formats 



Format 


Number 


Percent 


Both PDF and ASCII 


53 


21.9 


PDF only 


41 


16.9 


ASCII only 


25 


10.3 



SOURCE: National Commission on Libraries and Information Science, Government Information Product 
Assessment Questionnaire, 1998. 
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Public Access to Products Study Question 3: If a product is permanently accessible, is it also likely 

to be scheduled for retention with the National Archives and 
Records Administration (NAHA)? 

No, the majority of products surveyed that are permanently accessible 
are not likely to also be scheduled for permanent retention v^ith NARA. 

Permanent public accessibility and permanent record retention are two 
distinct concepts. GPO, through the FDLP, has a historical commitment to 
permanent accessibility of paper products, and now to electronic products. 
To that end, GPO requests that agencies provide information products in all 
mediums to GPO and work with GPO and Federal depository libraries to 
provide permanent public accessibility to electronic products. Agencies are 
responsible for transferring those products that are scheduled as permanent 
records to NARA. However, not all records that are scheduled for 
permanent retention by NARA are products within the scope of the FDLP. 
For such records, permanent public accessibility through the FDLP is not 
an issue. 

Of the respondents who said yes, the product is permanently accessible, and 
the respondents who reported their product is scheduled for retention with 
NARA, only 25 percent reported that the product is both permanently 
accessible and also scheduled for retention with NARA (table 35). The 
majority of products that are publicly accessible are not likely to also be 
scheduled for retention with NARA. While there is not information from 
the survey data to identify reasons for this situation, some possibilities are 
that: 

• the product is not a permanent or official record of the Lf.S. 
Government as defined by Federal Records legislation. 

• the product is in a format that is accepted by GPO but in a format that 
NARA does not currently accept, and therefore could not be transferred 
to NARA. 

• agencies are overlooking this important part of the information life 
cycle of electronic products. 



Table 35. 

Number and percent crosstabulations of products that are permanently public accessible and 



Permanent public access 


Product scheduled with NARA 


Total 


Yes 


No 


Don’t know 


Number 


Percent 


Number 


Percent 


Number 


Percent 


Number 


Percent 


Yes 


61 


25.2 


113 


46.7 


3 


1.2 


177 


73.1 


No 


21 


8.7 


41 


16.9 


3 


1.2 


65 


26.8 


Total 


82 


33.9 


154 


63.6 


6 


2.4 


242 


99.9 



NOTE; Percents may not add to 100 because of rounding. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product Assessment Questionnaire, 1998. 
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Study Question 4: Is the licensing of search and retrieval software likely 
to be a barrier to unrestricted public access? 

No, for the products surveyed, the licensing of commercial search and 
retrieval software by the agency does not appear to be a barrier to 
unrestricted (no fee) use. 

Of the respondents who reported that they license commercial search and 
retrieval software for their products, and those who reported that all users 
are charged a fee for the products, only 2 percent who license commercial 
search and retrieval software also charge a fee for all users (table 36). A 
slightly larger number of respondents (4 percent) who use commercial 
search retrieval software for their products also charge a fee for some 
users. Twenty-five percent of respondents who license search and retrieval 
software for their products charge no user fees. 



Table 36. 

Number and percent crosstabulations for products with licensed commercial search and retrieval 
software and user fees charged for the product 



Licensed commercial search and 
retrieval software 


User fees charged for product 


Total 


Yes, for all users 


No, for some users 


No 


Number 


Percent 


Number 


Percent 


Number 


Percent 


Number 


Percent 


Yes 


5 


2.1 


10 


4.1 


61 


25.2 


76 


31.4 


No 


16 


6.6 


38 


15.7 


112 


46.3 


166 


68.6 


Total 


21 


8.7 


48 


19.8 


173 


71.5 


242 


100.0 



NOTE: Percents may not add to 100 because of rounding. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product Assessment Questionnaire, 1998. 
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Study Question 5: Are respondents who have purchased commercial 
search and retrieval software for their products also transferring 
the products to NARA? 

No, based on the products surveyed here, respondents are not 
transferring permanent records to NARA for products in which they 
have purchased commercial search and retrieval software. 

Of the respondents who reported issuing commercial search and retrieval 
software, and those who reported scheduling products for permanent 
retention with NARA, only about 10 percent who have purchased 
commercial software for products have also scheduled their products for 
permanent retention with NARA (table 37). 



Table 37. 



Number and percent crosstabulations of those products with licensed commercial search 
and retrieval software and the product is scheduled for permanent retention by the 
National Archives and Records Administration (NARA) 



Licensed commercial search 
and retrieval software 


Product scheduled for permanent retention by NARA 


Total 


Yes 


No 


Don’t know 


Number 


Percent 


Number 


Percent 


Number 


Percent 


Number 


Percent 


Yes 


24 


9.9 


52 


21.5 


0 


0.0 


76 


31.4 


No 


58 


24.0 


102 


42.1 


6 


2.5 


166 


68.6 


Total 


82 


33.9 


154 


63.3 


6 


2.5 


242 


100.0 



NOTE: Percents may not add to 100 because of rounding. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product Assessment Questionnaire, 1998. 
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Other Issues: Authenticity Study Question 6: If an agency ensures authenticity, is it also likely to 
and Metadata provide permanent public access to the product or do agencies 

rely on another agency to provide permanent public access? 

Yes, based on the products surveyed, agency respondents who ensure 
authenticity for their products are also more likely to provide 
permanent access to them directly, rather than through another agency. 

Of those respondents who reported they ensure authenticity and those who 
reported they provide direct permanent public access to their products, 
47 percent both ensure authenticity for their products and provide direct 
permanent access to them (table 38). However, only close to 14 percent of 
the respondents who reported they ensure authenticity for their products 
also reported that another agency provides permanent public access to the 
product (table 39). 



Table 38. 

Number and percent crosstabulations of those products for which agencies 



Agency ensures authenticity 


Permanent public access 
provided by the agency 


Total 


Yes 


No 


Number 


Percent 


Number 


Percent 


Number 


Percent 


Yes 


113 


46,7 


41 


16,9 


154 


63,6 


No 


63 


26,0 


23 


9,5 


86 


35,5 


Don’t know 


1 


0,4 


1 


0,4 


2 


0,8 


Total 


177 


73.1 


65 


26.8 


242 


99.9 



NOTE: Percents may not add to 100 because of rounding, 

SOURCE: National Commission on Libraries and Information Science, Government Information Product Assessment Questionnaire, 1998, 



Table 39, 

Number and percent crosstabulations of those products for which 



Agency ensures authenticity 


Another agency provides 
permanent public access 


Total 


Yes 


No 


Number 


Percent 


Number 


Percent 


Number 


Percent 


Yes 


33 


13,6 


121 


50,0 


154 


63,6 


No 


17 


7.0 


69 


28.5 


86 


35,5 


Don’t know 


1 


0,4 


1 


0.4 


2 


0.8 


Total 


51 


21,0 


191 


78,9 


242 


99.9 



NOTE: Percents may not add to 100 because of rounding. 

SOURCE: National Commission on Libraries and Information Science, GovernnKnt Information Product Assessment Questionnaire, 1998, 
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Study Question 1: Are online products hosted by the agency that created 
it more likely to have a metadata record than products hosted by 
another agency? 



Yes, based on the products surveyed, those that are hosted by the 
agency that created it are more likely to have a metadata record than 
those hosted by another agency. 

Tables 40 and 41 show that almost 20 percent of the products that are 
hosted by an agency also have a metadata record, while only 7 percent of 
the products that are hosted by another agency also have a metadata record. 



Table 40. 



Number and percent crosstabulations of products that are hosted by the agency and have a metadata 
record 



Products hosted by the agency 


Metadata record 


Yes 


No 


Don't know 


Total 


Number 


Percent 


Number 


Percent 


Number 


Percent 


Number 


Percent 


Yes 


48 


19.8 


141 


58.3 


10 


4.1 


199 


82.2 


No 


17 


7.0 


24 


9.9 


1 


0.4 


42 


17.3 


Total 


65 


26.8 


165 


68.2 


11 


4.5 


241 


99.5 



NOTE: Percents may not add to 100 because of rounding. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product Assessment Questionnaire, 1998. 



Table 41. 



Number and percent crosstabulations of products that are hosted 
by another agency and have a metadata record 



Products hosted by another 
agency 


Metadata record 


Yes 


No 


Don’t know 


Total 


Number 


Percent 


Number 


Percent 


Number 


Percent 


Number 


Percent 


Yes 


17 


7.0 


25 


10.3 


0 


0.0 


42 


17.3 


No 


48 


19.8 


140 


57.9 


11 


4.5 


199 


82.2 


Total 


65 


26.8 


165 


68.2 


11 


4.5 


241 


99.5 



NOTE: Percents may not add to 100 because of rounding. 

SOURCE: National Commission on Libraries and Information Science, Government Information Product Assessment Questionnaire, 1998. 
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Qualitative 

Findings 



This section of the report highlights the qualitative findings from the three 
site visits with Federal depository libraries, five agency meetings, and six 
expert interviews. Appendices F through H include interview questions and 
detailed responses from the site visits to depository libraries (F), agency 
meetings (G), and expert interviews (H). 



Site Visits to Federal The purpose of the site visits to the three depository libraries was to identify 
DeDOSitorv Libraries issues and concerns librarians have about providing public access to 

^ ^ electronic Government information products through the Federal Depository 

Library Program. (See Appendix F for a complete list of questions posed to 
librarians.) 

The site visits were held with one regional depository library and two 
selective depository libraries in the Washington, D.C., metropolitan area. It 
is important to note that the three libraries visited may not be representative 
of all depository libraries in terms of the geographical location and library 
user characteristics (e.g., education level, socioeconomic status, etc.). 
Therefore, readers are cautioned about generalizing these observations to all 
depository libraries. Highlights of the three librarians’ responses are 
provided below. Appendix F contains a detailed description of the 
librarians’ responses to the interview questions. 



Highlights of Site Visits 
to Three Depository 
Libraries 



User Needs and Concerns 

• Librarians interviewed noted that the general public is still more 
comfortable using Government information products in paper and 
microfiche than they are using the Internet. Patrons (and librarians) are 
least comfortable using products on CD-ROM. 

« Librarians expressed concern about the difficulty patrons experience in 
accessing Government-produced CD-ROMs that are not standardized. 
They reported that the search and retrieval software is different for each 
CD, CD-ROMs often have no installation instructions or user 
documentation, and they are not user-friendly. 

• Librarians indicated that some users are still intimidated by electronic 
mediums and computers. Most users ask librarians to help them search 
for materials on the web and frequently need help downloading large 
files. 

• Librarians noted that since most Government websites only contain the 
most recent information, they are concerned about users having 
permanent public access to retrospective Government information on 
the web in the future. 
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Librarians’ Concerns: User Fees, Hardware, Training, and Costs 

• Although none of the libraries visited currently charge fees for printing 
materials from the Internet or CD-ROMs, all three librarians are either 
considering charging fees or are planning to charge fees and expressed 
concerns about how this will affect their patrons. 

• Users do not have access to enough workstations, so the libraries must 
limit use. Also, if libraries had additional money for hardware, they 
would order hardware in support of CD-ROMs (e.g., a new CD-ROM 
server and an 18-disk CD changer). (Even though CD-ROM is the 
least preferred medium and declining in number in the FDLP.) 

• All librarians interviewed expressed concerns about finding time and 
money to train librarians and staff, especially on using CD-ROM 
products, but also on downloading files, effectively searching the 
Internet for Government information, and creating and maintaining web 
pages. They welcome any additional training on using GPO Access, 
Geographic Information Systems, etc. 

• Time and money permitting, librarians expressed interest in establishing 
partnerships with GPO and other Government agencies to put some 
retrospective online Government information on their servers so users 
can have reliable access to it in the future. In addition, librarians would 
like to provide outreach to public schools, community centers, etc., to 
educate students and adults about the wide variety of valuable 
information available from the Federal Government. 

• One librarian expressed strong feelings about the need for Congress to 
provide long-term financial support to Federal depository libraries so 
they can continue to provide permanent public access to digital 
materials. This librarian’s perspective was that the cost to provide 
access to electronic Government information is steadily increasing. 



Agency Meetings 



Meetings were held with four agencies between September 14 through 
September 24, 1998: 

• Department of Health and Human Services 

• Environmental Protection Agency 

• U.S. Department of Education 

• U.S. Department of Commerce 



Although meetings also were held with the U.S. Supreme Court and the 
National Archives and Records Administration (NARA), these two agencies 
did not respond to the agency discussion questions in the agency meetings; 
they chose to discuss the survey questionnaire only. However, 
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Lewis Bellardo, the Deputy Archivist of the United States, sent in written 
responses to the discussion questions (Appendix G). 

The National Commission on Libraries and Information Science (NCLIS) 
provided discussion questions for the agency meetings; Westat modified 
some of the questions with NCLIS’s approval. The purpose of the agency 
meetings was to supplement survey data by collecting more general 
information on electronic Government information products that are not 
product-specific. For example, one of the survey objectives is to assess the 
cost-effectiveness and usefulness of preferred medium and format 
standards, an issue that was not directly addressed on the survey. In 
addition, the agency meetings afforded NCLIS, GPO, and Westat an 
opportunity to review the survey questionnaire with agency respondents and 
to address any questions they might have. 

Highlights of the agency meetings are provided below. For a more detailed 
summary of the responses to the 12 questions posed to agencies, see 
Appendix G. 



Agency Meeting Highlights Preferred Mediums and Formats 

• Agencies interviewed reported using the same preferred medium and 
format standards as those reported by survey respondents: web, CD- 
ROM, bulletin board; HTML, PDF, and ASCII. Additional preferred 
formats mentioned by agency representatives include TIFF, JPEG, and 
Lotus/Domino. 

• All agencies are exploring a wide range of innovative and creative web 
design approaches including the use of SQL, Oracle, ColdFusion, and 
animated GIFs. Some examples of ways in which agencies are utilizing 
web technologies include data warehousing, interactive GIS, 
multimedia CD-ROM, live “real-time” web casting of selected 
speeches, and real-time forecasting of air pollution levels for 22 states. 

• Four of the five agencies have guidelines or “best practices” for 
presentation and organization of products or publications on the web. 
Most of the guidelines discuss preferred formats for some types of 
products. The most common problem experienced by the agencies in 
this regard is compliance issues (i.e., encouraging personnel to adhere 
to them). 

• There are some trends for migrating certain families of products to the 
web for newsletters, training manuals, annual reports, and conference 
proceedings and presentations. 

• Agencies consider many factors when making decisions to create/retain 

products in more than one medium: budget, cost, accessibility to users, 
and size of audience the product reaches. The 
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decision-making process varies from agency to agency and sub-unit to 
sub-unit. 



Assessing User Needs 

• All agencies reported involving users in testing and evaluating the 
usefulness of the web and CD-ROM products. The most frequently 
used assessment methods are focus groups, videotaping of users, and 
online user surveys. Agencies are using the results of these evaluation 
methods to add and change some formats and mediums as well as 
content. 

• Four of the five agencies interviewed reported that they maintain some 
type of GILS records to help the public locate their information 
resources. 

Information Life Cycle Management, Permanent Public Access, and 

Permanent Retention 

• No agencies are addressing the following key information resources 
management issues: permanent public access, information life cycle 
management, and permanent retention. (The expert interviews provide 
some insight into the reasons that agencies are not addressing these 
issues. See the summary section of this report.) 

Cost-Effectiveness of Various Mediums and Formats 

• No agencies have conducted a formal cost-benefit analysis for creating 
products in formats and mediums for distribution to the Federal 
Depository Library Program. Generally, agency representatives 
reported it costs less to create products for the web because they can 
avoid production, printing, and distribution costs for paper and CD- 
ROM products. 



Expert Interviews interviews with six experts also enriched and supplemented the survey 

findings. Since the interviews were conducted after the site visits and 
agency meetings, they were helpful in providing a broad context within 
which the survey findings could be viewed. 

The expert interviews were conducted between October 27 and November 
24, 1998. Telephone interviews were held with two webmasters, two 
preservation specialists, and two professors of information resources 
management. These experts were selected from a list provided by the 
NCLIS. Highlights from each set of interviews are provided here. (See 
Appendix H for a detailed summary of each telephone interview.) 
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Interviews With 
Webmasters 



Interviews With 
Preservation Specialists 







Highlights from interview with webmasters Jerry Malitz, National Center 

for Education Statistics (NCES), and Linda Wallace^ the Internal Revenue 

Service (IRS), on October 27, 1998. 

Preferred Formats 

• The IRS, unlike the other agencies surveyed, primarily uses SGML, 
followed by PDF, HTML, and Postscript. They train their authors to 
use SGML because they consider it ‘‘intelligent data” that can 
automatically generate other formats (e.g., web, BBS, Fax on Demand) 
through templates and filters. All NCES publications are in PDF, then 
HTML (optional); they rarely put an entire publication in HTML 
format only. 

• The IRS has conducted a cost-benefit analysis of the costs of delivering 
requests through different formats. They have found that it costs $3 per 
phone call to fill a request, 1 cent to access their Internet site for forms, 
etc., and $2.50 to make a CD-ROM containing 5 years of IRS 
publications. 

• IRS indicates that it provides permanent public access to tax 
information online for 5 years, and from their “core repository library” 
for about 14 years, but not for every application. However, this 5 to 14 
years means that IRS provides current but not permanent public access 
to their Government electronic products. 

• All IRS documents are ADA-compliant, online searchable, and 
downloadable. 

User Needs 

• Both IRS and NCES assess and evaluate the effectiveness of their web 
sites with advisory groups (IRS), or for NCES, through an Internet 
Working Group made up of representatives from, each program area. 

• Both agencies have GILS records. 



Highlights from interview with preservation specialists Evelyn Frangakis 
from the National Agricultural Library (NAL) and Abby Smith from the 
Council on Libraries and Information Resources (CLIR), November 10, 
1998. 

Goals of Preservation 

• It is useful to think about preservation goals such as enhancing the 
long-term preservation of and access to information of enduring value 
for as long into the future as possible. 

• There is no standard accepted method of ensuring long-term access to 
digital information. It may be more accurate to say that one of the 
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primary goals of preservation is to set up systems that “sustain 
predictable levels of loss.” 

Barriers to Preservation of Digital Materials 

• The concept of preservation in the traditional preservation world 
examines the concept of permanence, but in the print world the concept 
of permanence relates to chemical inertness and mechanical durability. 
These concepts do not translate easily into a digital world. 

• There are two problems with digital preservation: (1) media in which 
information resides may be unstable; and (2) sofilware/hardware 
configurations on which information is stored becomes obsolete so 
quickly that even when one migrates information from one system to 
another, much of the data and functionality are lost. 

• Other barriers to digital preservation include that it is difficult to 
understand what we can and cannot do under current copyright law, 
and any transmission link is as strong as the weakest link. The weak 
link in the transmission of electronic information is human beings, not 
technology (e.g., no one agency or organization has stepped forward to 
address issues like information life cycle management). Preservation of 
information must be thought about at the creation stage, not after the 
information has been collected and disseminated. 

• One of the core infrastructure problems is the need to create a failsafe 
archives mechanism for materials that disappear from the web. 

Current Preservation Models and Initiatives 

• NAL and partner institutions are implementing a model for permanent 
public access and preservation of agricultural literature that addresses 
all the key issues in information resources management: inventory and 
life cycle of information, permanent public access, technical 
requirements, and user access and retrieval. (NAL is one of the few 
examples for ensuring a failsafe archives for preservation of 
agricultural literature.) 

CLIR Initiatives 

• CLIR commissioned a report by Jeff Rothenberg from RAND 
Corporation on emulation. (Emulation is the process of imitating one 
system with another so both accept the same data, execute the same 
programs, and achieve the same results.) 

• CLIR commissioned an analysis of migrating file formats to do a risk 
assessment associated with those file formats during migration. 
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Interviews With 
Information Resources 
Management Specialists 







• CLIR identified a computer scientist at Carnegie Mellon University, 
John Ockerbloom, who has developed a system of file conversion called 
TOM (Typed Object Model), a type of migration that converts web- 
based materials to different file formats. 



Highlights from interviews with John Bertot, November 18, and Charles 

McClure, November 28, 1998, (These two telephone interviews were held 

separately,) 

Barriers to Successful Implementation of Information Resources 

Management Initiatives 

• Agencies are struggling with issues such as permanent public access, 
information life cycle management, and permanent retention due to a 
general lack of information resources management (IRM), as well as 
organizational policy integration for Federal Government legislation 
and initiatives. 

• Agencies do not view information as a strategic resource that is directly 
related to agency missions. Most Government IRM initiatives focus on 
the technology side of IRM because it is tangible. 

• Sometimes smaller agencies are more successful in implementing IRM 
initiatives due to fewer organizational and communication barriers to 
working collaboratively. 

• The Information Technology Management and Reform Act of 1996 did 
little to clarify the role of the CIO and IRM staff, so agencies are now 
struggling with what to do with these functions. 

• Agency resources are now almost exclusively devoted to Y2K efforts 
with little time and resources left to devote to IRM, standards, and 
operability. 

• Staffing and training are critical for both IRM and CIO staff. 

® Challenges for agencies in the next few years include how to coordinate 
information technology and information technology management, 
interoperability and standards that cut across agencies, and education 
and training of staff. 
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Discussion of 
Quantitative 
and 

Qualitative 

Findings 



This section synthesizes, integrates, and discusses issues and the major 
themes that emerge from the survey and the qualitative data collection 
activities, including the interviews with Federal depository librarians, 
agency personnel, and other experts, and the literature review. The section 
is arranged by the following key study issues: 

• Preferred mediums and formats, 

• Evaluating websites. 



• Cost-effectiveness of formats and mediums, 

• Depository library needs. 



• Public access (public domain and user fees), 

• Permanent public access and preservation. 



Preferred Mediums 
and Format 
Standards 



Survey respondents and agency representatives reported they most often use 
the following mediums: 

• Paper 

• Web 

• CD-ROM 



• Bulletin board systems (to a lesser degree) 



Both respondents and representatives also reported use of the following 
formats: 

• HTML 

• PDF 

• GIF 

• ASCII 

• TIFF 



However, most agencies whose products were surveyed use these mediums 
and formats as a common agency practice, rather than as an agency 
mandate. In addition, agency representatives and webmasters reported they 
use SGML, Oracle (with ColdFusion or SQL), JPEG, and TIFF because 
these formats meet the information needs of their individual constituents or 
are used in some of their creative web approaches. The IRS is one of the 
few agencies interviewed that uses SGML. IRS’ Linda Wallace, one of the 
webmasters who served as an expert consultant for this project, indicates 
that most agencies do not use SGML because it is difficult to use. But 
Wallace noted that IRS uses SGML because it is 
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much more robust, and it is easy to change a document format to match 
customer needs (e.g., tax law information for consumers and for lawyers). 
(See Appendix H for detailed notes on the telephone interview with Linda 
Wallace.) 

A few survey respondents indicated they are planning to change to or add 
XML or other object-oriented formats. XML may be appealing to some 
agencies because data can be stored in a format provided by XML that is 
transferable to a wide range of hardware and software environments 
(Bryan, 1998, p. 14). In addition, according to Stuart Culshaw, XML 
makes it easier for authors to produce documents for many different output 
mediums (i.e., paper, online help, web) from a single source (Culshaw, 
1998, p. 7). 

Most of the agency representatives who participated in the meetings also 
reported that their agencies have established written guidelines or “best 
practices” that specify preferred formats for the presentation of information 
on the web. Even though these guidelines are not agency-mandated, they 
seem to be a common agency practice. Several of the agencies interviewed 
indicated they have modified or adopted their agency guidelines from the 
guidelines established by the Federal Web Consortium in 1996. 

The Consortium, founded in 1994 by the National Science Foundation and 
the U.S. Nuclear Regulatory Commission, established guidelines with other 
Government agencies (see http://www.dtic.mil/staff/cthomps/ guidelines/). 
The guidelines provide suggestions to help the Federal community 
accomplish agency missions to improve services to customers. Consortium 
guidelines cover a wide range of topics including: 

• Home page checklist (content, navigation/organization, style/markup); 

• File formats (i.e., agencies should not be restricted to proprietary 
formats such as WordPerfect, Microsoft Word, SAS, PDF); 

• Rationale for using certain kinds of formats such as HTML, GIF, and 
JPEG; 

• Guidelines for formats to be used for downloading or display (e.g., 
HTML, GIF, JPEG, PDF, Postscript); and 

• Emerging standards. 

Agency representatives indicated that one of their biggest challenges is to 
convince personnel from all program areas to follow the agency’s internal 
guidelines when creating products for the web. Another challenge for 
agencies is to consolidate web guidelines from different agency sub-units so 
they are complementary rather than contradictory. 
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Evaluating 

Websites 



Agency representatives, per 0MB Circular A- 130 and the Government 
Performance and Results Act of 1993, are assessing the usefulness of their 
websites and CD-ROMs as part of a larger effort to measure program 
effectiveness. Focus groups, online customer surveys, and videotaping of 
customers online are the most common ways in which agencies evaluate and 
test products on their websites. 

One objective of the evaluation is to test both formats and web approaches. 
Based on the evaluation results, agencies may change or add formats. For 
example, one agency, after testing their site with children, eliminated PDF 
files on the site and made it more interactive. Another agency made the 
decision to keep their BBS because many of their international users do not 
have ready access to the web. One agency webmaster indicated that the 
needs of their business clients, who participate on their advisory board, help 
drive their format needs. A fourth agency stores its documents in TIFF 
format for image and textual data. As customers request documents, the 
agency converts them to PDF so customers can download the material. A 
fifth agency created a simple set of rules for producing CD-ROMs based 
upon user input: keep it simple to use, intuitive, and self-tutorial. 

User needs for easy access to electronic information products will continue 
to affect how agencies make decisions about formats and mediums. Bertot 
and McClure suggest that more agencies should continue to monitor the 
information needs of the public as well as targeted constituencies to enhance 
current access to electronic Government information products (Bertot and 
McClure, 1997, p. 288). 



Cost-Effectiveness 
of Formats 
and jViediums 



None of the agency representatives who attended the agency meetings has 
conducted a formal cost-benefit analyses for producing or creating products 
in preferred or emerging formats, mediums, or online approaches for 
distribution to the FDLP. Most agencies reported that migrating products 
to the web substantially reduces printing and distribution costs associated 
with paper mediums. However, the crosstabs in tables 28-29 reveal that 
many Government products are still produced in more than one medium and 
often in more than one format. Providing permanent public access to 
electronic mediums ultimately may exceed the one-time costs associated 
with producing and distributing the same information in print or microform 
(GPO, 1996, p. 24 and A71-A74). 



In her role as Chief, Electronic Information Services, at the Internal 
Revenue Service, Linda Wallace has analyzed the costs of delivering 
documents to customers (Appendix H). She found that: 



It costs IRS $3 per call for the public to call into their toll-free number 
and for IRS to fill the request. 
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• The cost to IRS for the public to use the Internet to access and use the 
forms is 1 cent, a difference of 300 to 1. (However, this shifts the cost 
to the public, who must have access to the Internet.) 

• It costs IRS $2.50 to make and distribute to all public libraries 
(including the Federal depository libraries) each CD-ROM containing 5 
years of tax forms, instructions, and publications. 

Based on these numbers, the IRS has made some internal decisions about 
where they will focus their resources and time in order to reach the 
maximum number of customers in the most cost-effective manner. 



Depository Library 
Needs 



Since depository librarians serve as the intermediary between the users and 
electronic information products, their observations and experiences about 
user and library needs are critical. In general, the five agencies interviewed 
focused on public users or their target audiences rather than depository 
library users when discussing usage of their electronic Government 
information products. 



First, the librarians interviewed emphasized that many patrons still prefer 
Government information in paper mediums, followed by the web and then 
CD-ROM. The respondents surveyed indicated that many of their products 
are produced both in paper and on the web. 



Second, librarians expressed concerns about lack of standardization for 
producing Government CD-ROMs. One agency representative indicated 
that they are undertaking several initiatives to make their CD-ROMs more 
user-friendly by making them as intuitive as possible and incorporating a 
user testing component into the production schedule. 



A third important concern for the librarians interviewed is the rising cost of 
computer hardware and the simultaneous rise in user expectations for state- 
of-the-art computer workstations. Although the three libraries recently 
received updated computer workstations that met or exceeded the 
recommended minimum guidelines for depository libraries, they are 
beginning to change their policies on access to workstations by placing a 
time limit on their use. 



A fourth issue concerns the rising costs to purchase and maintain new 
equipment, which have caused depository librarians to reconsider their 
policies on charging printing fees. One librarian indicated that their library 
already charges patrons for photocopying materials; this change is not 
dramatic, but it does affect the concept of no-fee access when an 
overwhelming number of products are offered on the Internet. 

Fifth, time and resources to train library staff (and patrons) on how to use 
the new technology (i.e., how to download files), conduct Internet searches, 
design and develop their own websites, and load, search, and use 
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Public Access 



Permanent Public Access 
to and Permanent 
Retention of Electronic 
Government Information 



O 




CD-ROMs are major concerns expressed by the depository librarians 
interviewed. The fact that Government information exists in a variety of 
mediums and formats only increases rather than diminishes the need for 
training. 

Finally, all librarians are troubled by how GPO, the FDLP, and 
Government agencies will address the problems of permanent public access 
to electronic information products that are constantly being replaced and 
updated by new ones. In addition, the preservation of retrospective 
electronic Government information is an issue of concern. 



The survey data revealed that 1 5 percent of the products surveyed are not in 
the public domain, for all or part of the product. In addition, user fees are 
charged for 30 percent of the products. These data suggest that these two 
critical public access goals have not yet been achieved. 



Perhaps more than any other issues, permanent public access and 
preservation pose two of the greatest challenges to the FDLP, and 
ultimately to the public. Each of the experts raised different issues and 
shared various perspectives about these issues. It might be helpful here to 
summarize their perspectives and describe initiatives underway to address 
the problems associated with the provision of permanent public access and 
preservation. 

Most of the survey respondents indicated that permanent access is currently 
provided for the products surveyed, although most of the responses 
indicated that this concept is not fully understood and that access is not 
provided by the agency responsible for the product. Instead, they are 
relying on GPO, Federal depository libraries, the National Technical 
Information Service, or other agencies to provide this permanent public 
access. In its policy and planning document. Managing the FDLP 
Electronic Collection (see http://www.access.gpo.gov/ 

su docs/dpos/ecplan.html), GPO states that “the ‘first-level’ collection 
management activity depends upon knowledge that the products exist. In 
order to ensure current and permanent access, GPO will ...rely on 
notification from and outreach to other agencies and notification from the 
depository library community.” 

The responses of agency representatives on the issue of permanent public 
access may provide additional information about the problem. Most 
agency representatives said their agencies had not discussed the issue or 
were exploring the issue to see how it should be addressed, and they 
indicated that they did not understand the concept of permanent public 
access in relation to permanent retention. The one exception was the 
representative from National Archives and Records Administration, who is 
clear about the agency’s role to provide permanent public access to its own 
products. 
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It might be helpful here to clarify the distinctions between the two concepts. 
GPO’s definition of permanent public access “means that electronic 
Government information products within the scope of the FDLP remain 
available for continuous, no-fee public access through the program” (GPO, 
1998, p. 19). 

Lewis Ballardo, deputy archivist of the United States, in a recent article in 
the Washington Post (March 12, 1999, p. AOl) stated that the problem of 
digital preservation must be addressed “or memory will be lost for the latter 
half of the 20^*^ century.” In addition, Bellardo, in a written response to 
agency questions, articulated agency responsibilities to GPO for permanent 
public access and to NARA for permanent retention. GPO will accept 
products in all mediums to provide continuous, no-fee public access, if 
notified by agencies that access is being discontinued. Agencies are 
responsible for transferring those products that are scheduled as permanent 
records (official records as defined by Federal Records legislation) to 
NARA. 

Linda Wallace described the IRS’ methods for providing current public 
access to their materials. Using SGML format, the IRS has built and 
maintains a core knowledge repository to generate media output in any 
application to respond to customer needs. The repository maintains 
materials for 14 years, but not for every application. In addition, all tax 
forms, publications, instructional materials, etc., are available online for 5 
years. Since none of the agencies interviewed is providing permanent 
public access to its products, it was useful to ask two information resources 
management experts, John Bertot and Charles McClure, to provide some 
larger context within which the problem can be viewed. 



Perspectives on Permanent 
Public Access and 
Information Life Cycle 
Management from 
Information Resources 
Management Experts 



Both Bertot and McClure have extensively studied and taught information 
resources management (IRM). They attribute the lack of successful 
implementation of IRM initiatives in the Federal Government to the 
following factors: 

® There is no comprehensive integrated Federal IRM policy; current 
policies do not adequately address permanent public access, 
information life cycle, and electronic records management. 



• There is no strategic vision of IRM by agencies; information is not 
viewed as a resource that should be used to accomplish agency 
missions. 



• Most agency initiatives focus on the technology side of IRM because it 
is tangible. 

• Most agencies are targeting their information technology resources 
toward Y2K efforts. 



There is no clear distinction between the role of information resources 
managers and CIOs. 
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• There is no ongoing training for IRM and CIO staff. 

(See Appendix H for detailed notes on telephone interviews with Bertot and 
McClure, and Bertot and McClure, 1997, pp. 280-282.) 

There are many IRM policy instruments from the Paperwork Reduction Act 
of 1980 and 1986, 0MB Circular A-130 (1985; and 1993 and 1994 
revisions) through the Information Technology Management Reform Act 
(ITMRA) of 1996 and Executive Order 13011 (July 1996). But Bertot and 
McClure (1998) emphasize that there is still a lack of an integrated policy. 
For example, in their focus group with IRM managers, Bertot and McClure 
noted that managers felt that the Paperwork Reduction Act assumed that the 
managers understood and knew how to manage the information life cycle, 
but they agreed that agency management at all levels never grasped the 
concept either in theory or in practice. In addition, the ITMRA that created 
a position for an agency-based CIO to oversee agency IRM activities and to 
provide education for agency IRM personnel and agency managers (among 
other things) does not clarify the relationships between and among CIOs 
and IRM managers. Consequently, it is ambiguous about whether the 
agency CIO’s organization replaces, incorporates, or is separate from 
current agency IRM functions. 

Given this larger context, it is not surprising that IRM issues such as 
information life cycle management, preservation, and permanent public 
access have not been adequately addressed. Conventional organizational 
barriers such as size, culture, poor communication and interaction across 
and within agencies, and lack of ongoing, strategic training for IRM and 
CIO staff may exacerbate these challenges faced by agencies (telephone 
interview with Bertot, Appendix H). (As an example, McClure states that 
IRM graduate students’ degrees are useful for about 1-2 years after they 
graduate. After that, their skills are 50 percent out of date; telephone 
interview with McClure, Appendix H.) Several experts are involved in 
initiatives that address some of these important IRM issues. 



Current Initiatives on Several agencies, organizations, and Federal depository libraries with 

Permanent Public Access partner institutions are exploring ways to address the problems of 

and Permanent Retention permanent public access, preservation, and electronic records management. 

Appendix H contains more detailed information about each of these 
initiatives that will be summarized here. 

Abby Smith from the Council on Libraries and Information Resources 
(CLIR), and Evelyn Frangakis from the National Agricultural Library 
(NAL) are supporting research and testing models for permanent public 
access and preservation. The three CLIR initiatives are described below: 
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• A commissioned report by Jeff Rothenberg from RAND Corporation on 

emulation. The report has been completed and was published 
in January 1999. The report describes the weaknesses of migration and 
the strengths of emulation and sets up a research agenda to develop 
emulation. (Log onto publications on CLIR site for a summary of 
Rothenberg’s report: http://www.clir.org/pubs/reports/ 

rothenberg/contents . html. ) 

• A commissioned analysis of migrating file formats to support a risk 
assessment associated with those file formats during migration. The 
study by Cornell University, using data from the Mann (agricultural) 
Library, will use numeric file formats and databases and text formats. 
The report, to be finished by September 1999, will include analysis and 
a template that others can use for doing a risk assessment of migration 
of those file formats. 

• CLIR is working with John Ockerbloom, a computer scientist at 
Carnegie Mellon University (CMU) who has developed a system of file 
conversion called TOM (Typed Object Model). (See www.es. 
cmu.edu/afs/cs.cmu.edu/user/spok/wvm/defense/index.html). CLIR 
would like to see if they can bring his concepts into fuller application. 

Smith describes NAL and its efforts to provide permanent public access and 
to preserve agricultural literature as one of few examples where a failsafe 
archives might work, partly because NAL is a national library dedicated to 
one type of literature. Evelyn Frangakis is involved in NAL’s efforts to 
develop its ovm preservation program that includes a traditional 
preservation program and digital efforts. Their digital efforts are two- 
pronged: 

• Conversion of brittle paper materials into digital products by working 
with the best available guidelines to implement good preservation 
practices. They will make this digital material available on the web. 

• Development of a program to preserve USDA digital materials (i.e., 
materials that are born digitally). 

In addition, Frangakis is also involved in a national effort to preserve 
agricultural literature. The U.S. Department of Agriculture’s Digital 
Publications Preservation Steering Committee was established in 1998 to 
oversee the implementation of the plan, A Framework for the Preservation 
of and Permanent Public Access to USDA Digital Publications. This 
group met for the first time in October 1998. The plan may serve as a 
model that other agencies or institutions can adapt. USDA is incorporating 
the following needs and considerations into its framework: 

• Inventory and life cycle management, 

• Technical requirements, and 
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Next Steps 
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• User access and retrieval. 

USDA is moving ahead to implement the plan. The USDA CIO accepted 
the report, and under Frangakis’ guidance, NAL established a national 
steering committee made up of representatives from USDA and from 
agribusiness, the research library community, the U.S. Agricultural 
Information Network (USAIN), Federal partners, etc. The group will meet 
on a quarterly basis for the first 2 years. They will establish test groups to 
explore the technical and funding issues. They are hoping to secure funding 
for a pilot project to test the framework on an agency within USDA to see 
how manageable it will be for frill-scale implementation (see Appendix H 
for a detailed description of the Framework). 

Finally, GPO has established partnerships with several depository libraries 
and Federal agencies to provide permanent public access to remotely 
accessible electronic Government information products. Three such 
partnerships include: 

• Partnership with the University of Illinois at Chicago’s Richard J. 
Daley Library and the U.S. Department of State (DOS) to provide 
permanent access to remotely accessible electronic DOS information 
products. 

• An Online Computer Library Center/GPO pilot project with the U.S. 
Department of Education/National Library of Education (NLE) 
provided free public access through the FDLP to remotely accessible 
electronic Educational Resources Information Center (ERIC) 
documents. 

• A project with the Department of Energy (DOE) Office of Scientific 
and Technical Information (OSTI) to provide public and depository 
library access to DOE technical reports in image format via the web 
service called “DOE Information Bridge” (Aldrich, 1998), 



Preservation specialists Smith and Frangakis noted that technology is not 
the biggest barrier to permanent access and preservation; the human 
infrastructure is not in place yet that would ensure permanent access and 
preservation (telephone interview with Smith and Frangakis, Appendix H). 
The plans and initiatives described here, coupled with the recommendations 
for training, policy integration, and support for best practices to implement 
policies are a few of the strategic actions that appropriate agencies, 
libraries, and institutions should undertake to ensure that future generations 
will have unrestricted, no-fee access to Government information in all 
formats. 



As a followup effort, NCLIS indicated that they will use these findings as a 
point of departure and analyze them in greater depth. It is expected that 
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this followup effort will result in broad conclusions and recommendations to 
the President and Congress about how the problems and challenges revealed 
in this study can be constructively addressed. 
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Agency Study Coordinator Meetings Agenda 



Assessment of Electronic Government Information Products 
Agency Study Coordinator Meetings 
July 23, 1998, and August 4, 1998 

The Benton Foundation 
1634 I Street, N.W. 

11“* Floor 
Washington, D.C. 



Agenda 



9:30 - 9:45 Background and Welcome 

Robert S. Willard, Executive Director, National Commission on Libraries 
and Information Science (NCLIS) 

Francis J. Buckley, Jr., Superintendent of Documents, Government Printing Office (GPO) 

9:45 - 10:15 Objectives, Timetable, and Required Actions 
Forest Woody Horton, NCLIS Study Manager 

10:15-10:30 Coffee Break 



10:30 — 12:15 Specific Data Requirements (question-by-question review of survey instrument using 
GPO Access as a specific product example) 

Gil Baldwin, GPO 
TC. Evans, GPO 
Ric Davis, GPO 
Denise Glover, Westat 
Elizabeth Farris, Westat 
Steve Fischer, Westat 



12:15 



12:30 



Wrap-up and Closing 
Forest Woody Horton 

Adjourn 
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Agency Coordinators and Other Key Officials 



Agency 


Agency Coordinator(s) 
Legislative Branch 


Library of Congress 


Nancy Davenport 
Maggie Smith 


United States Congress 


Eric Peterson 


Administrative Office of the U.S. Courts 


Judicial Branch 

Gloria Malkin 


Supreme Court of the United States 


Shelley L. Dowling 
Wilma M. Grant 


Department of Agriculture 


Executive Branch 

Betty Behai 


Department of Commerce 


Vera Whisenton 
Cynthia Banicki 


Department of Defense 


William Beyer 
Rick Silva 


Department of Education 


Chiquitta Thomas 
Linda Tague 


Department of Energy 


Karen Spence 



Department of Health and Human Services Deborah Burris 





Fred Wood 


Department of the Interior 


Claude Christensen 
Bob Mehnert 


Department of Justice 


Regina Byrd 
Chris Rudy 
Dennis Feldt 



Department of Labor 



Maureen Hill 
Deborah Klein 



Agency 



Agency Coordinator(s) 



Executive Branch (continued) 



Department of State 


Colleen Hope 
Dan Clemmer 


Department of Transportation 


Barbara Post 
Robert Zarnetske 


Department of the Treasury 


Gladys Myatt 
Mike Conklin 


Environmental Protection Agency 


Richard Huffine 


Executive Office of the President 


Peter Weiss 


General Services Administration 


Odessa Brown 


National Aeronautics and Space Administration 


Roland Ridgeway 


National Archives and Records Administration 


Debra Leahy 


Securities and Exchange Commission 


Bert Lee 


Smithsonian Institution 


Robert Schelin 


Social Security Administration 


Terry Hynes 
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List of Participating Agencies and Products Surveyed 



Participating Agencies and Products Surveyed 



Survey 

status 



Legislative Branch 



Library of Congress 

American Memory: Historical Collections CM 

Braille Books on Disk CM 

Cataloger's Desktop CM 

CD BLND CM 

Classification Plus CM 

Country Studies CM 

French Poster Art CM 

THOMAS CM 

Thomas: A Century of Lawmaking for a New Nation CM 

Thomas: Bill Summary and Status CM 

Thomas: Committee Reports CM 

Thomas: Congressional Record CM 

Thomas: Early Congressional Documents CM 

Thomas: Enactment of a Law CM 

Thomas: Federalist Papers CM 

Thomas: Floor Activities for the House & Senate CM 

Thomas: House Committee Schedules OS 

Thomas: House Roll Call Votes CM 

Thomas: How Our Laws are Made CM 

Thomas: Major Legislation CM 

Thomas: Senate Roll Call Votes CM 

Thomas: Text of Bills CM 



United States Congress 
Congressional Bills 
Congressional Directory 
Congressional P.ecord (bound 
Congressional Record (daily) 
Hearings, Reports, & Prints: 
Hearings, Reports, & Prints: 
Hearings, Reports, & Prints: 
Hearings, Reports, & Prints: 
Hearings, Reports, & Prints: 
Hearings, Reports, & Prints: 
Hearings, Reports, & Prints: 
Hearings, Reports, & Prints: 
Hearings, Reports, & Prints: 
Hearings, Reports, & Prints: 



r\ TV» o n f ^ 
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House Agriculture Committee 

House Appropriations Committee 

House Education and Workforce Committee 

House Government Reform and Oversight Committee 

House Judiciary Committee 

Joint Committee on Taxation 

Senate Appropriations Committee 

Senate Armed Services Committee 

Senate Commerce Committee 

Senate Foreign Relations Committee 



CM 

CM 

A 

CM 

♦ 

♦ 

CM 

CM 

CM 



♦Survey nonrespondent. 
CF = Complete by fax 
CM = Complete by mail 
CP = Complete by phone 
OS = Out-of-scope 



Survey 

status 

Legislative Branch (continued) 

United States Congress (continued) 

Hearings, Reports, & Prints: Senate Judiciary Committee CM 

Hearings, Reports, & Prints: Rules and Administration Committee * 

Serial Set * 

United States Code CM 

Judicial Branch 

Administrative Offlce of the U.S. Courts 

Federal Rules of Civil Procedures CM 

Judicial Business of the United States Courts CM 

The Official Bankruptcy Forms Collection CM 

The Third Branch CM 

Understanding the Federal Courts CM 

Supreme Court of the United States 

Bench Opinions of the Supreme Court CM 

Rules of the Supreme Court CM 

Slip Opinions of the Supreme Court CM 

U.S. Reports CM 

Executive Branch 

Department of Agriculture 

Agent Orange CM 

AgExporter CM 

AGRICOLA Database * 

Agricultural Prices, Monthly CM 

Agricultural Statistics CM 

Aquaculture CM 

Continuing Survey of Food Intake By Individuals CM 

Crops County Data CM 

Fact Sheets CM 

FAS Hot Country Pages CM 

Fire Effects Information System CM 

Forest Land Distribution Data for the U. S. CM 

Leaflets CM 

Market News CF 

NAFTA Agricultural Fact Sheets CM 

Ornamental Horticultural Multimedia Project CM 



♦Survey nonrespondent. 
CF = Complete by fax 
CM = Complete by mail 
CP = Complete by phone 
OS = Out-of-scope 
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Executive Branch (continued) 



Survey 

status 



Department of Agriculture (continued) 

Outlook for U.S. Agricultural Exports CM 

Statistical Bulletins CM 

The Plant Genome Database Collaboration CM 

Wildland Fire Assessment System CM 



World Agricultural Supply and Demand Estimates 



Department of Commerce 

AgroBase Database * 

Cen-Data OS 

CenStats * 

Census of Population and Housing * 

Commerce Business Daily/CBDNet CM 

County and City Data Book CM 

Current Population Reports * 

Economic Census Reports * 



Federal Research in Progress Database 
Geophysics of North America 



GOV.Research-Center * 

Imports/Exports CD CM 

Local Climatalogical Data (for states) CM 

National Trade Data Bank CM 

NTIS Database * 

Regional Economic Information System (REIS) CM 

Solar Radio Bursts * 

Statistical Abstract of the United States CM 

STAT-USA/Internet/State of the Nation CM 

ST AT-USA Newsletter CM 

Qlinx/ox/ r\'f* T5i i c iriocc {^\ /f 

wUllVllt X^VAi3111Vi3i3 V«/1V1 

TIGER/Line CM 

U.S. Industry and Trade Outlook CM 

USA Counties CM 

Zip Code Area CD-ROM CM 

Department of Defense 

Airman Magazine CM 

Airpower Journal CM 

All Hands * 

Defense Logistics Agency Publishing System CM 

Joint Electronic Library CM 



♦Survey nonrespondent. 
CF = Complete by fax 
CM = Complete by mail 
CP = Complete by phone 
OS = Out-of-scope 






Executive Branch (continued) 



Survey 

status 



Department of Defense (continued) 

Marines Magazine CM 

Notice to Mariners CM 

Soldiers Magazine CM 

Technical Manuals CM 

Department of Education 

Condition of Education CM 

Digest of Education Statistics CM 

Disability Statistics Abstract CM 

Disability Statistics Report CM 

EDsearch, Education Statistics on Disk CM 

ERIC Database CM 

Guide to Education Programs CM 

Helping Your Child (series) CM 

NAEP (Nat'l Assessment of Educational Progress) Mathematics CM 

National Education Goals Report CM 

Projections of Education Statistics CM 

Resources in Education CM 

Student Guide CM 

Think College? Me? Now? CM 



Department of Energy 

Country Analysis Briefs (CABs) * 

DOE Directives * 

DOE Information Bridge CM 

DOE R&D Project CM 

Electric Power Annual CM 

Electric Power Monthly CM 

Emissions of Greenhouse Gases in the U.S. * 

Energy Consumption & Carbon Emissions by Region * 

Energy Files CM 

Energy InfoDisc CM 

Energy Science and Technology Database CM 

International Energy Outlook CM 

Monthly Energy Outlook * 

National Education Goals Report * 

Petroleum Supply Annual CM 

State Energy Data Reports CM 

State Energy Data System CM 




* Survey nonrespondent. 
CF = Conriplete by fax 
CM = Complete by mail 
CP = Complete by phone 
OS = Out-of-scope 
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Executive Branch (continued) 



Survey 

status 



Department of Energy (continued) 


Weekly Petroleum Status Report 


CM 


World Energy Consumption 


* 


Department of Health and Human Services 


Annual Pesticide Residue Monitoring Report 


CM 


Biosafety in Microbiological and Biomedical Laboratories 


CM 


CDC Wonder 


CM 


Clinical Preventive Services 


CM 


EHP (Environmental Health Perspectives) 


CM 


FDA Almanac 


CM 


FDA Compliance Policy Guide 


CM 


HCFA Health Watch 


CM 


HCFA*s Laws, Regulations, Manuals 


* 


Healthcare Cost & Utilization Project (HCUP-3) 


CM 


Medicare and You 


CM 


Medicare Compare 


CM 


Morbitity & Mortality Weekly report 


CM 


National Health Interview Survey, State Data Files 


CM 


NIOSHTIC Database 


CM 


PubMed 


CM 


Research Activities (Monthly Newsletter) 


CM 


RTECS Database 


CM 


SETS: Statistical Export & Tabulation System 


CM 


Vital and Health Statistics (Rainbow Series) 


CM 


Department of the Interior 


Contaminant Information Mgt & Analysis System 


CM 


PnHsmgArpH Rnllpfin 


r-A/f 


Geographic Names Information System 


CM 


Metal Industry Indicators (Mil) 


CM 


Mineral Industry Surveys (monthly & quarterly) 


CM 


Minerals Yearbook 


CM 


National Survey of Fishing, Hunting & Wildlife & 


* 


National Wildlife Refuge System Profiles Database 


CM 


New Publications of the Geological Survey 


CM 


NPS Statistical Abstract 


CM 


Preservation Briefs (numbered series) 


* 


The National Register Information System (NRJS) 


CM 


Water Resources Abstracts 


CM 




♦Survey nonrespondent. 
CF = Complete by fax 
CM = Complete by mail 
CP = Complete by phone 
OS = Out-of-scope 



C-7 



nr; 

w U 



Executive Branch (continued) 



Survey 

status 



Department of Justice 

Correctional Populations of the United States CF 

FBI Law Enforcement Bulletin CM 

FOIA Annual Report CM 

Justice Information Center Publications CM 

NCJRS Abstracts Database CM 

Prison and Jail Inmates CF 

Sourcebook of Criminal Justice Statistics CF 

Uniform Crime Reports: Crime in the United States CM 

Department of Labor 

General Wage Determinations Issued Under Davis-Bacon * 

Monthly Labor Review CM 

Occupational Outlook Handbook CM 

OSHA Documents and Files * 

Department of State 

Background Notes CM 

Diplomatic List * 

Dispatch CM 

Key Officers of Foreign Service Posts * 

Maximum Travel Per Diem Allowance for Foreign Area * 

Medical Information for Americans Traveling Abroad CF 

Treaties and Other International Acts Series * 

Department of Transportation 

FAA Statistical Handbook CM 

Highway Statistics CM 

National Transit Database CM 

National Transportation Statistics CM 

Nationwide Personal Transportation Survey CM 

Rail Waybill Data CM 

Transportation Expressions CM 

Transportation Statistics Annual Report CM 

Worldwide Transportation Directory CM 

Department of Treasury 

A Visitors Guide OS 

Arson Investigations Guide, ATF P 2220.1 * 

Bonded Warehouse-Manual for Proprietors, Importers & Customs Officers CM 




♦Survey nonrespondent. 
CF = Complete by fax 
CM = Complete by mail 
CP = Complete by phone 
OS = Out-of-scope 
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Survey 

status 

Executive Branch (continued) 

Department of Treasury (continued) 

Business Taxpayers Information Publications, Pub 1 194.B CM 

Buying Treasury Securities, PD P 0009 * 

Counterfeiting & Forgery * 

Customs Valuation Encyclopedia, 1980-1997 * 

Daily Treasury Statements CM 

Federal Tax Forms CM 

Firearms State Laws & Published Ordinances, ATF P 5300.5 CM 

Importing a Car (Pub 520) CM 

Know Your Money * 

Payment of Tax by EFT, ATF P 5000. 1 0 CM 

Quarterly Journal (Online) CF 

Tables for Redemption Values for US Series E Savings Bonds & Savings Notes CM 

Tables of Redemption Values for US Series EE Savings Bonds CM 

Tax Guide for Small Businesses, Pub 334 * 

The History of Money OS 

The Money Story CF 

Treasury Bulletin * 

U.S. Mint Gift Collection CF 

Your Federal Income Tax, Pub 17 CM 

Your Guide to Federal Firearms Regulations, ATF P 5300.4 * 

Environmental Protection Agency 

Center for Environmental Info. & Statistics CM 

Consumer Handbook for Reducing Solid Waste CM 

Envirofacts Warehouse CM 

EPA Online Library System (OLS) CP 

Federal Register Environmental Subset CM 

Guide to Environmental Issues CM 

Integrated Risk Information System Database (IRIS) CP 

National Air Quality and Emission Trends Report CM 

National Environmental Publications Information CM 

National Water Quality Inventory: Biennial Report to Congress (305b report) CM 

Recycle City CM 

Sector Facility Indexing Project CM 

Setting the Record Straight: Secondhand Smoke is a Preventable Health Risk CM 

Superfimd Hazardous Waste Site Query (CERCLIS Data) CM 

Surf Your Watershed CM 

Test Methods for Evaluating Solid Waste: Physical/Chemical Methods (SW-846) CM 



♦Survey nonrespondent. 
CF = Complete by fax 
CM = Complete by mail 
CP = Complete by phone 
OS = Out-of-scope 
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Survey 

status 

Executive Branch (continued) 

Executive Office of the President 

Art in the White House -- A Nation's Pride * 

Best Kept Secrets in Government CM 

Budget of the United States * 

FBIS Publications Reports * 

From Red Tape to Results; Creating a Government & CM 

Interactive Citizens' Handbook * 

NAICS Manual * 

0MB Circulars 

Putting Customers 1st, Serving the American Public CM 

Reinvention Express CM 

The White House for Kids 

Virtual Library * 

White House Briefing Room * 

White House History (web) * 

White House Tour (CD-ROM) * 

World Factbook CM 

World News Connection * 

General Services Administration 

Catalog of Federal Domestic Assistance CM 

Consumer Information Catalog CF 

Consumer Information Series OS 

Consumer Resource Handbook CM 

Federal Acquisition Regulation * 

Government Registration Service CM 

MarkeTips * 

U.S. Gold CM 

U.S. Government TDD/TTY Directory CM 

United States Online Directories 

US Government Blue Pages Online Directory CM 

US Real Property Sales List (online title varies) CM 

National Aeronautics and Space Administration 

Aerospace Medicine and Biology CM 

CASI Technical Report Server CM 

NASA Thesaurus CM 

NASA Video Catalog CM 

Patent Abstracts CM 

Scientific, Technical Aerospace Reports CM 

♦Survey nonrespondent. 

CF = Complete by fax 
CM = Complete by mail 
CP = Complete by phone 
OS = Out-of-scope 
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Executive Branch (continued) 



Survey 

status 



National Archives and Records Administration 

Code of Federal Regulations 

Digital Classroom 

Emerging Nation 

Federal Register 

National Archives Information Locator (NAIL) 
Online Exhibit Hall 
Privacy Act Issuances 
Public Laws (slip laws) 

United States Government Manual 

Weekly Compilation of Presidential Documents 



CM 

CM 

CM 

CM 

CM 

CM 

CM 

CM 

CM 

CM 



Survey 

status 



Securities and Exchange Commission 
Administrative Proceedings 



CM 



Cold Calling * 

Commission Legal Briefs * 

Completed Initiatives of Interest to Small Business * 

Directory of Companies Filing Annual Reports * 

EDGAR Database of Corporate Information * 

Financial Facts Tool Kit * 

Investment Fraud & Abuse Travel to Cyberspace * 

Litigation Actions and Proceedings Bulletin OS 

Litigation Releases CM 

Official List of Section 13 (f) Securities * 

Official Summary of Security Transactions & Holdings * 

Pending Initiatives of Interest to Small Business * 

Plain English Handbook: How to Create Clear SEC * 

SEC Annual Report * 

SEC Concept and Interpretative Releases CM 

SEC Decisions CM 

SEC Docket OS 

SEC Final Rules CM 

SEC News Digest (daily) CM 

SEC Opinions CM 

SEC Proposed Rules CM 

SEC Special Studies CM 

Staff Accounting Bulletins CM 

Trading Suspensions CM 



♦Survey nonrespondent. 
CF = Complete by fax 
CM = Complete by mail 
CP = Complete by phone 
OS = Out-of-scope 
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Survey 

status 



Smithsonian Institution 

Annals of the Smithsonian Institution CM 

Anthology of American Folk Music CM 

Gallery Exhibition Catalogues * 

SIRIS: SI Research Information System CM 

Smithsonian (magazine) * 

Smithsonian Contributions to Anthropology CM 

Smithsonian Contributions to Botany CM 

Smithsonian Contributions to Earth Sciences CM 

Smithsonian Contributions to Marine Sciences CM 

Smithsonian Contributions to Paleobiology CM 

Smithsonian Contributions to Zoology CM 

Smithsonian Folklife Festival: Culture of, by & . . . * 

Smithsonian Folklife Studies Series CM 

Smithsonian Institute Research Reports * 

Smithsonian Photographs Online * 

Smithsonian Studies in Air and Space CM 

Smithsonian Year * 

Social Security Administration 

Documents Published by the SSA Historian CM 

Publications Information Pamphlets and Fax Sheets CM 

Request a Personal Earnings & Benefits System CM 

SSA Publications on CD-ROM CM 



♦Survey nonrespondent. 
CF = Complete by fax 
CM = Complete by mail 
CP = Complete by phone 
OS = Out-of- scope 
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Coordinator and Respondent Cover Letters 



United Slates 
National Commission on 
LibreiriCsS and Information Science 



October 1998 



Dear Agency Coordinator: 

On behalf of the National Commission on Libraries and Information Science (NCLIS) and the Government 
Printing Office, we would like to thank you for agreeing to distribute this survey of Government Electronic 
Information Products to agency respondents. 

The purpose of this assessment is to: (1) identify medium and format standards that are the most 
appropriate for permanent public access, (2) assess the cost-effectiveness and usefulness of alternative 
medium and format standards, and (3) identify public and private medium and format standards that are or 
could be used for products throughout their entire information life cycle. The broader objective of the 
survey is to study the long-term impacts of shifting government information products from paper and 
microform mediums to federal agency web sites and other electronic mediums. We want to ensure that as 
more government information is available in a variety of electronic mediums and formats, the American 
public continues to have free and easy local access to this information through the Federal Depository 
Library Program. 

We are requesting that you, as the agency coordinator, promptly distribute the questionnaires to the 
appropriate personnel in your agency. If you have any questions about this survey, please call Denise 
Glover at Westat: 301-25 1-2269 or toll-free at 800-937-8281, ext. 2269. We will send you a copy of the 
final report once it is completed. 

The following suggested procedures and instructions will assist you to successfully distribute the product 
questionnaires. However, some agency coordinators have decided to use different data collection 
procedures. If you are one of those coordinators, it is essential that you explain your procedures to 
your respondents. 

1 . All agency coordinators will receive a full packet from Westat on September 29 or shortly thereafter 
that contains a cover letter and an information copy of the questionnaire. The coordinator's packet will 
also include a sealed packet for each product. Please make sure you have a packet for each product on 
your final list of product selections. If you are missing information or have incorrect information, 
please call Debbie Alexander at Westat: 301-294-2088, or toll-Iree at 800-937-8281, ext. 2088. 

2. The product packet includes a cover letter to the respondent, a product questionnaire, a glossary of 
terms, and a postage-paid return envelope. 

3. Each product questionnaire contains the following preprinted information: a five-digit ID, the agency's 
name and submit, and the name of the product. 

4. Upon receipt of the packet, please distribute the product packets to the appropriate product respondents 
in your agency. 

1 1 lO Vermont Avenue, N.W. Suite 820 
Washington, D.C. 20005-3522 
(202) 606-9200 
Fax; (202) G06-9203 
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5. Due to the specialized nature of many of the questions asked on the survey, it may be necessary for 
product respondents to consult with other agency personnel such as records managers, information 
technology staff, planning offices, and others to complete the survey in its entirety. We recommend 
that you ask product respondents to leave blank any questions they feel uncomfortable answering and 
to make arrangements for someone in their office or another office with more appropriate knowledge 
and expertise to answer those questions. If product respondents decide to take this action, we strongly 
suggest that, if possible, you standardize the arrangements in advance and ensure that all agency 
respondents are aware of your procedures. One purpose of standardizing arrangements is to ensure 
that product respondents do not lose control of a questionnaire because multiple individuals and 
offices are handling it 

6. Product respondents are responsible for ensuring that all questions are answered on their respective 
questionnaires, even if they must consult with other personnel. 

7. To ensure consistency and completeness, you might want to ask your assigned product respondents to 
allow you to review the surveys before they send them to Westat. You might also request that 
respondents send you a copy of the completed questionnaire for your files. 

8. Product respondents should return completed questionnaires directly to Westat (not to you) in the 
enclosed postage-paid envelope. If product questionnaires are not completed and returned to Westat by 
the October 30, 1998 deadline, Westat will attempt to follow up directly with the designated product 
respondent. However, if respondent information is unavailable or unknown, Westat will contact you 
for followoip. 

Respondents are requested to return the survey by October 30, 1998 by mailing it in the postage-paid 

envelope to: 

Denise Glover 
TA 2064 
Westat, Inc. 

1650 Research Boulevard 
Rockville, MD 20850 

Thank you very much for your assistance and cooperation. We appreciate the time and hard work you have 

invested in the coordination activities. 



Sincerely, 





Robert S. Willard 
Executive Director 

National Commission on Libraries and Information Science 



Francis J. Buckley, Jr. 
Superintendent of Documents 
Government Printing Office 



United Slates 
National Commission on 
Libraries and Information Science 



January, 1999 



Dear Respondent: 

On behalf of the National Commission on Libraries and Information Science (NCLIS) and the Government 
Printing Office, we are requesting that you complete this survey of Government Electronic Information 
Products. 

The purpose of this assessment is to: (1) identify medium and format standards that are the most 

appropriate for permanent public access, (2) assess the cost-effectiveness and usefulness of alternative 
medium and format standards, and (3) identify public and private medium and format standards that are or 
could be used for products throughout their entire information life cycle. The broader objective of the 
survey is to study the long-term impacts of shifting government information products from paper and 
microform mediums to federal agency web sites and other electronic mediums. We want to ensure that as 
more government information is available in a variety of electronic mediums and formats, the American 
public continues to have free and easy local access to this information through the Federal Depository 
Library Program. 

We recognize that many respondents will complete surveys for more than one agency product. We also 
understand the burden this imposes upon you and appreciate the time and effort you will devote to 
completing the survey. However, we want to underscore the importance of your participation. We will 
send you a copy of the report summarizing the data from these surveys once it is completed. 

Please carefully read all of the following suggested procedures and instructions that will assist you to 
successfully complete and return the product questionnaires. However, some agency coordinators have 
decided to use different data collection procedures. You should contact your coordinator directly to 
find out if you are to follow the procedures outlined here. If your coordinator has made other 
arrangements, please contact him/her to determine what they are. 



1 . Once you receive your packet(s) from your agency coordinator, please check to make sure each 
packet contains the following information for each product you are to survey: a cover letter, 
questionnaire, glossary of terms, and postage-paid return envelope. If you feel you are missing 
information or have incorrect information, please first contact your agency coordinator. (Your 
coordinator's contact information is available on the NCLIS web site at www.nclis.gov . I If 
your coordinator is unable to provide this information, call Debbie Alexander at Westat: 301- 
294-1088, or toll-free at 800-937-8281, ext. 2088. 

2. Each product questionnaire contains the following preprinted information: a five-digit ID, the 
agency's name and subunit, and the name of the product. 



1 1 lO Vermont Avenue. N.W. Suite 820 
Washington, D.C. 20005-3522 
(202) 606-9200 
Fax; (202) 006-9203 




Fedwal Recycling Program 0 
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Pfinted on Recycled Paper 

1.15 



3. Due to the specialized nature of many of the questions asked on the survey, it may be necessary 
for you to consult with other agency personnel such as records managers, information 
technology staff, planning officers, and others to complete the survey in its entirety. We 
suggest you leave blank the responses to questions you feel uncomfortable answering and then 
arrange for someone in your office or another office with more appropriate knowledge and 
expertise to answer those questions. Your agency coordinator may have standardized those 
arrangements in advance and made you are aware of their procedures. The purpose of 
standardizing arrangements is to ensure that you do not lose control of a questionnaire 
because multiple individuals and offices are handling it. 

4. Please note that you are responsible for ensuring that all questions are answered on your 
respective questionnaires, even if you must consult with other personnel. 

5. To ensure consistency and accuracy, coordinators might request that they be allowed to review 
the completed surveys before you return them to Westat. 

6. Return completed questionnaires directly to Westat in the enclosed postage-paid envelope by 
January 15, 1999. Your coordinator may request that you also send him/her a copy of the 
completed questionnaire. 

7. If you have any questions about this survey, please call Denise Glover at Westat: 301-251- 
2269, or toll-free at 800-937-8281, ext. 2269. 

Please return the questioiuiaire by January 15, 1999 by mailing it to: 

Denise Glover 

TA 2064 

Westat, Inc. 

1650 Research Blvd. 

Rockville, MD 20850 

Thank you very much for your assistance and cooperation. 



Sincerely, 





Robert S. Willard 
Executive Director 

National Commission on Libraries and Information Science 



Francis J. Buckley, Jr. 
Superintendent of Documents 
Government Printing Office 



Appendix £ 

Questionnaire and Glossary of Terms 



NATIONAL COMMISSION ON LIBRARIES AND INFORMATION SCIENCE 



GOVERNMENT INFORMATION 
PRODUCT ASSESSMENT QUESTIONNAIRE 



Directions: This form is to be completed only for products that are either already in electronic mediums or products that are 
to be migrated to electronic mediums. Do not complete this form for products that will remain in paper or microform 
mediums only. A product is defined as “a Government publication or other work of the United States Government conveyed 
in a tangible physical medium such as a book, CD-ROM, etc., or disseminated through an electronic Government information 
service and intended for public dissemination.” (See enclosed glossary for definitions of terms used throughout this 
questionnaire.) Complete one questionnaire for each product. (Please note that a Website is not considered a product, 
although products might be on a Website.) 



A. GENERAL INFORMATION 

1. Agency Name: 

Sub-Unit: 

2. Name of Product: 

3. Brief Description of Product: _ 



4. URL for Product Website: 

CH Check if no Website 

B. CURRENT PRODUCT PROFILE 

5. How is this product used by the end user? (Check all that apply.) 

a. Information access and retrieval □ ‘ 

b. Data analysis (e.g., to support analysis by end user) □ * 

c. Other (specify) Q * 



What types of data are contained within this product? 

In Column A, indicate which type of data this product contains. (Check all that apply.) 

In Column B, indicate the primary type of data contained in this product. For example, if you checked boxes for 
items b and f in column A, indicate which of the two is the primary type of data by placing a check in the appropriate 
box in column B. 



Type of data 


A. Type of data 
contained 


B. Primary 
type of data 
(Check only 
one) 


(Check all that apply) 


a. Bibliographic data 


□' 


□' 


b. Graphical data (photos, charts, graphs, tables, drawings) 


□' 




c. Numerical data 


□' 




d. Sound 


□' 




e. Spatial data (maps, coordinate files) 


□' 




f. Textual data (books, serials, reports) 


□' 




g. Video 


□' 




h. Multimedia (sound, video, text, graphics) 


□' 




i. Other {specify) 


□' 
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7a. In what mediums is this product publicly available? 

In Column A, indicate which type of medium is used. (Check all that apply,) 

In Column B, for each medium used, indicate whether there is a medium standard that is mandated by the agency, a 
common practice in the agency (although not mandated), other (i.e., not agency-mandated standard or common 
agency practice, but new and promising and beginning to be used, etc.), OR none of these. 



Medium 


A. Type of 
mediums used 


B. Standard — 
Is there... 


C. Primary type 


(Check all that 
apply) 


Agency- 

mandated 

standard 


Common 

agency 

practice 


Other 


None 


of medium used 
(Check only one) 


(Check one for each catesorv) 


' Pre-Electronic Mediums 














a. Paper 












□' 


b. Microform 










n'* 




c. Other (specify) 








1 1 


1 1 


1 1 


Electronic Mediums 














Magnetic Mediums 














d. Magnetic tape 


□ ' 












e. Floppy diskette 


□ ' 


□' 








n’ 


f. Hard drive 










1 1 


1 1 


g. Other {specify) 




□' 




1 1 


1 1 


[ 1 


Describe medium in more detail 














Optical Mediums 














h. CD-ROM 










ri'* 


n* 


i. WORM (Write once, read many disk) 


□ ' 


□' 




1 1 


1 1 


1 t 


j. DVD (digital video disk) 










ri'* 


|—|10 


k. Other (specify) 




□’ 






1 1 


1 1 


Describe medium in more detail 














Online Mediums 














1. Web 














m. Gopher 


□ ' 


□' 










n. Bulletin Board Systems 












□ ■4 


0 . Other (specify) 














Describe medium in more detail 















7b. If you checked “Other’’ category in Question 7a, Column B, please explain. 
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8a. Which of the following types of formats does this product use? 

In Column A, indicate which type of format is used. (Check all that apply.) 

In Column B, for each format used, indicate whether there is a format standard that is mandated by the agency, a 
common practice in the agency although not mandated, other (i.e., not agency-mandated standard or common agency 
practice, but new and promising and beginning to be used, etc.), OR none of these. 

In Column C, for each type of format used, check one box to indicate the primary type that is used within each 
major category (e.g., database, spreadsheet, word processing, etc.). 





A. Formats 
used? 




B. Standard — 
Is there... 




C. Primary type 
of format used 
(Check one in 
each category) 


Format 


(Check all that 
oddIv) 


Agency- 

mandated 

standard 


Common 

agency 

practice 


Other 


None 




**rr*J/ 


(Check one for 


each category) 




Database 

a. Oracle 


□ ' 


□' 








□' 


b. Sybase 


□ ' 


□' 










c. dBase 


□ ' 


□' 










d. WAIS 


□ ' 


□' 










e. MARC 


□ ' 


□' 










f. Other (specify) 


□ ' 


□' 










'Spreadsheet _ 












. .. : 


a. Excel 


□ ' 








□ 4- ■ 


□ ' 


b. Lotus 1-2-3 


□ ' 


□' 










c. Other (specify) 


□ ' 


□' 










Tagge d Mar f^p 

a. HTML 


□' " 


□' 









□' 


b. XML 


□ ' 


□' 










c. SGML 


□ ' 


□' 










d. Other (specify) 


□ ' 


□' 










Mintage 






^ 

’ 







" ( 


a. GIF 


□ ' 








□ 


b JPEG 


□ ' 


□ ' 










c TIFF 


□ ' 


□ ' 










d PDF 1 


□ ' 


□ ' 










e. Other (specify) 


□ ' 


□ ' 










Audio _ 

a WAV 




□ ' 




" 




□ ' 


b AU 


□ ' 


□ ' 










c AIFF 


□ ' 


□ ' 










d. Other (specify) 


□ ' 


□ ' 










; Video ' 














a. MOV 


□' 




□ 2 








b. MPEG 


□ ' 


□ ' 










c AVI 


□ ' 


□ ' 










d. Other (specify) 


□ ' 


□ ' 










'Text 

a. ASCII 


□' 


□ ' 








□ ' 


b. Rich Text Format 


□ ' 


□ ' 










c. ANSI 


□ ' 


□ ' 










d. Other (specify) 


□ ' 


□ ' 













8a. (continued) 



Format 


A. Formats 
used? 


B. Standard — 
Is there... 


C. Primary type 
of format used 
(Check one in 
each category) 


(Check all that 
apply) 


Agency- 

mandated 

standard 


Common 

agency 

practice 


Other 


None 


(Check one for each catesorv) 




Wordfypc^sing^^^^^ L . 

a. Word Perfect 


□' 

□ ' 

□ ' 




□' 

□' 


ct"' 


- - □a'-' 




1 

□' 


b. Microsoft Word 


c. Other (specify) 


, Other (specify) 












□’ i 


•• — - 



8b. If you checked “Other” category in Question 8a, Column B, please explain. 
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9a. Is this product in an online medium? 

Yes n (Continue with question 9b) No D (Skip to question 10a) 

9b. Which of the following online approaches are used? 

In Column A, indicate which type of approach is used. (Check all that apply,) 

In Column B, if the approach is used, indicate whether it is mandated by the agency, a common practice in the agency 
although not mandated, other (i.e., not agency-mandated standard or common agency practice, but new and promising 
and beginning to be used, etc.), OR none of these. 





A. Type of on- 
line tool used 




B. Standard — 
Is there... 




Online approaches 


(Check all that 
aDplv) 


Agency- 

mandated 

standard 


Common 

agency 

practice 


Other 


None 






(Check one for 


each categor 





User Interfaces Supported 

a. Netscape (specify version) 


□' 


□' 








h Internet Explorer (specify version) 


□ ' 


□' 








c. Telnet 


□ ' 


□' 








d FTP 


□ ' 


□' 








e. Nongraphical/dial-up shell 


□' 


□' 








f. Other (specify) 


□' 


□' 








\ Web Design Approaches 

a. Rasic HTML onlv (specify version) 


□' 


□' 




' 




b Table*' 


□' 


□' 








c. Frames 


□ ' 


□' 








d. CGI Scripts 


□' 


□' 








e. Use of Javascript 


□' 


□' 








f. Use of Java Applets 


□ ' 


□' 








g XML 


□ ' 


□' 








h. Other (specify) 




□' 








Bulletin Board Systems (BBS) 












a. Graphical interface/browser 


□' 


□' 









9c. If you checked “Other” category in Question 9b, Column B, please explain. 
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Searchability of Product 

10a. Please indicate whether this product is... (Check all that apply,) 



a. Included as part of a full-text searchable database with no fielding □ ‘ 

b. Indexed by full-text and field Q ‘ 

c. Available as “view only” — non-searchable Q • 

d. Other (specify) Q * 



10b. This product is officially hosted by... (Check all that apply,) (Host refers to the primary site where the public can 
find the product,) 



a. Your agency Q* 

b. Another agency {specify) 

c. Contractor Q * 

d. Educational institution Q * 

e. Other {specify) Q • 



Retrievability 

1 1 . This product and any associated software. . . (Check all that apply,) 

a. Can be downloaded, saved, and is not subject to any restrictions on use or 

re-use by the end user Q • 

b. Cannot be downloaded, saved, and/or re-used because it is part of a database 

and does not exist as a distinct product □ ‘ 

c. Cannot be downloaded, saved, and/or re-used because it requires proprietary 

software that is not freely distributable □ ‘ 

d. Other (specify) Q • 



C. PLANNED PRODUCT PROFILE {This next section refers to future plans for the product,) 

Type(s) of Data - Future Plans 

12a. Are there any plans to discontinue publication of this product? 

Yes Cy (Continue with question 12b) No Cy (Skip to question 13a) 

1 2b. If yes, please explain. (Skip to Section D,) 



13a. What kind of data will this product contain? If product contains more than one type of data, respond for all data types. 
(Check one,) 

Retain existing type(s) of data, no changes planned □ ‘ (Skip to question 14a) 

Retain existing type(s) of data and add items of one or more new types of data 

(specify) (Continue with question 13b) 

Discontinue one or more types {specify) {Continue with question 13b) 

Change to new type(s) of data (specify) 

ini'* (Continue with question 13b) 

13b. Change(s) will occur in the: (Check all that apply,) 

Short term: within 1 year or less Q ’ 

Medium term: within 2 to 5 years Q * 

No changes indicated □ • (Skip to question 14a) 



If you checked both ‘‘short term” a/f^“medium term” in question 13b, continue with question 13c. Otherwise, skip to 
question 13d. 



ERIC 



1 
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13c. If you have short-term and medium-term plans, how are your short-term plans different from your medium-term 
plans? For example... (Check all that apply.) 

Short-term plans call for one type of data, but medium-term plans call for a different type of data. 

(specify) d 

In the medium term there are plans to combine resources to create a new version of a product using a 
different type of data that may be different from short-term plans. 

(specify) d 

Other (specify) d * 



13d. Do you have any long-term plans (6 or more years) for changing this product to new types of data? 

Yes d ‘ (Continue with question I3e) No (Skip to question 1 4a) 

13e. If yes, please describe them here. 



Medium - Future Plans 



14a. What kind of medium(s) will this product use? If product is delivered in more than one medium, respond for all 
mediums. (Check one.) 

Retain existing medium(s), no changes planned d ’ (Skip to question 1 5a) 

Retain existing medium(s) and add items of one or more new types of mediums 

(specify) d^ (Continue with question 14b) 

Discontinue one or more types {specify) d^ {Continue with question 14b) 

Change to new type(s) of medium 

(specify) d"^ (Continue with question 14b) 



No agency-mandated medium applies d^ (Continue with question 14b) 

14b. Change(s) will occur in the: (Check all that apply.) 

Short term: within 1 year or less d ' 

Medium term: within 2 to 5 years d ' 

No changes indicated d ’ (Skip to question 15a) 

If you checked both “short term” a/i//“ medium term” in question 14b, continue with question 14c. Otherwise, skip to 
question 14d. 
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14c. If you have short-term and medium-term plans, how are your short-term plans different than your medium-term 
plans? For example. . . (Check all that apply.) 

Short-term plans call for one type of medium, but the medium-term plans call for a different type 
of medium, (specify) Q * 

In the medium term there are plans to combine resources to create a new version of a product in a 
medium that may be different from short-term plans. 

(specify) □ ’ 

Other (specify) Q * 



14d. Do you have any long-term plans (6 or more years) for changing mediums for this product? 

Yes EH* (Continue with question 14e) No (Skip to question 15a) 

14e. If yes, please describe them here. 



Format - Future Plans 

15a. What kind of format(s) will this product contain? If the product uses more than one format, respond for all formats. 
(Check one.) 

Same as existing format(s), no changes planned D ’ (Skip to question 16a) 

Retain existing format(s) and add one or more new format types 

(specify) (Continue with question 15b) 



Change to new format type(s) (specify) 



(Continue with question 15b) 



Discontinue one or more types {specify) {Continue with question 15b) 

No agency-mandated format applies (Skip to with question 16a) 

1 5b. Change(s) will occur in the: (Check all that apply.) 

Short term: within 1 year or less O * 

Medium term: within 2 to 5 years ED* 

No changes indicated ED * (Skip to question 16a) 

If you checked “short term” a/i£^“ medium term” in question 15b, continue with question 15c. Otherwise, skip to 

question 15d. 




, fa u 
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15c. If you have short-term and medium-term plans, how are your short-term plans different from your medium-term 

plans? For example... (Check all that apply,) 

Short-term plans call for one kind of format, but the medium-term plans call for a 

different type of format, (specify) □ * 



In the medium term there are plans to combine resources to create a new version of a product using a 
different format that may be different from short-term plans. 

(specify) 



Oihtx (specify) ^ 



15d. Do you have any long-term plans (6 or more years) for changing formats for this product? 

Yes O' (Continue with question 15e) No (Skip to question 1 6a) 

15e. If yes, please describe them here. 



D. OTHER INFORMATION 
Metadata 

16a. Is there a metadata record for this product (e.g., GILS, MARC)? 

Yes CD' (Continue with question 16b) No CD^ (Skip to question 1 7 a) 

16b. If yes, please specify 



Permanent Access 



17a. 



Permanent public access to this product is currently provided by: (Check all that apply) 



Your agency 

Another agency {specify) 

Other {specify) 

No permanent public access provided... 



CD' (Continue to question 1 7b) 
CD' (Continue to question 17b) 
CD' (Continue to question 17b) 
CD' (Skip to question 17c) 



17b. How is permanent public access provided? (specify) 



(Go to question 18a) 



17c. 



Are there plans to provide permanent public access in the future for this product? 
Yes □' No 



Permanent Retention 

18a. Is this product scheduled for permanent retention by the National Archives and Records Administration? 

Yes CD' (Continue with question 1 8b) No CD^ (Skip to question 19a) 

18b. What is the planned retention period for this product? (specify) 
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Ensuring Authenticity 

19a. Does the agency ensure authenticity (official status determination) for this product? 

Yes (Continue with question 19b) No (Skip to question 20) 

19b. How does the agency attest to authenticity? (specify) 



Updating/Upgrading Plans 

20. How frequently is this product updated or refreshed? (Check one.) 



Daily 

Weekly Q2 

Monthly 

Annually 

Other (specif’) Q ^ 

2 1 a. Are there plans for changing the product’s supporting technology? 

Yes (Continue with question 2 1 b) No (Skip to question 22a) 



21b. If yes, specify 



User Fees 

22a. Is a user fee charged for this product? {Check one . ) 



Yes, for all users □ * {Continue with question 22b) 

Yes, for some users (Continue with question 22b) 

No (Skip to question 23a) 



22b. If yes, explain and specify amount of fees. 



Licensing 

23a. Has the agency licensed commercial search and retrieval software for use with this product? 

Yes □ ‘ (Continue with question 23b) No [3^ (Skip to question 24a) 

23b. Specify the software vendor and product name. 

23c. Does the agency’s license cover use by. . . (Check all that apply.) 



a. Agency personnel □ * 

b. Agency’s primary target constituencies □ ‘ 

c. Federal Depository Libraries Q ^ 

d. All libraries □ ’ 

e. Public users Q ‘ 

f Other 



Jl. <<• 
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Public Domain 

24a. Is this product in the public domain? (Check one.) 



Yes, for the entire product 

Yes, for part of product 

No 



n ^ {Continue with question 25) 
{Continue with question 24b) 
(Skip to question 24c) 



24b. If yes for part of product, please explain. 

24c. If no, has the agency entered into an arrangement with the private sector that would limit use of this information? 
(Please briefly explain the arrangement.) 



E. COMMENTS 

25. If you wish to comment on matters that you believe are not otherwise adequately covered in this survey, do so here. 



26. Key person completing this form. 

Contact Name: 

Title: 

Telephone: Fax: 

E-Mail: 

Other person(s) providing responses to questions or assistance in completing this form. 

27. Contact Name: 

Title: 

Telephone: Fax: 

E-Mail: 
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28. Contact Name: 

Title: 

Telephone: Fax: 

E-Mail: 



THANK YOU. PLEASE RETURN THIS SURVEY TO: 

DENISE GLOVER 
WEST AT, ROOM TA2064 
1650 RESEARCH BOULEVARD 
ROCKVILLE, MD 20850 
FAX: 301-517-4134 
PHONE: 301-251-2269 
glove rd 1 @we$tat.com 



FOR QUESTIONS ABOUT COMPLETING THE SURVEY, CONTACT DENISE GLOVER. 
PLEASE KEEP A COPY OF THIS SURVEY FOR YOUR RECORDS. 
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Glossary of Terms, Abbreviations, and Acronyms for the 
Government Information Product Assessment Questionnaire 



ASCII — An acronym for American Standard Code for Information Exchange, ASCII is an international standard 
in which numbers, letters, punctuation marks, symbols, and control codes are assigned numbers from 0 to 127. 

AIFF — Short for Audio Interchange File Format, a common format for storing and transmitting sampled sound. 

ANSI — Acronym for the American National Standards Institute, a voluntary organization that creates standards 
for the computer industry. In addition to programming languages, ANSI sets standards for a wide range of 
technical areas, from electrical specifications to communications protocols. 

AU — Short for audio, a common format for sound files on UNIX machines. 

AVI — A format developed by Microsoft Corporation for storing video and audio information. 

Accessibility — The degree to which the public is able to retrieve or obtain Government information products, 
either through the Federal Depository Library Program (FDLP) or directly through an electronic information 
service established and maintained by a Government agency or its authorized agent. The other aspects of 
accessibility include the degree to which these Government information products are available to the public in a 
useful format or medium and in a time frame in which the information has utility. In the context of the FDLP, 
accessibility includes the degree to which Government information is accurately identified and described 
bibliographically, the information’s availability is made known to the public, and technological, social, economic, 
political and physical barriers to gaining access are minimized. 

Authentication — The certification of a Government information product attesting to its legitimate official status. 
Ensuring the authenticity of a product involves product design, planning, and policy development, as well as 
technical considerations. 

Availability — ^The degree to which information is physically or electronically obtainable through the intentional 
or unintentional provision of Government information products to the public. In the context of the FDLP, 
availability includes the measures taken by Government agencies and the FDLP to include Government 
information products in the program. Accessibility is meaningless if information is unobtainable from its source. 

BBS — Bulletin Board System, an electronic message center. Most bulletin boards serve specific interest groups. 
They allow users to dial in with a modem, review messages left by others, and leave their own message. 

CGI Scripts — Abbreviation for Common Gateway Interface, a specification for transferring information between 
a World Wide Web server and a CGI program. CGI programs are the most common way for Web servers to 
interact dynamically with users. An increasingly common way to provide dynamic feedback for Web users is to 
include scripts or programs that run on the user's machine rather than the Web server. 

CD-ROM — Compact Disk-Read Only Memory; an optical disk from which information may be read but not 
written. 

DVD — Short for digital versatile disk or digital video disk, a new type of CD-ROM that holds a minimum of 
4.7GB (gigabytes), enough for a full-length movie. 

Digital Image — An electronic version of a bit-mapped image of a document or other information format that 
allows text to be searched at the character level; "digitalize" means the process and accompanying technologies 
required to effect the conversion from bit-mapped (e.g., a fax) to searchable format. 
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Dissemination — ^The act of making Government information products accessible to the public through 
distribution to depository or program libraries or by using a Government electronic information service. 

Electronic Government Information — Information that is organized, stored, and disseminated using electronic 
or optical mediums as opposed to paper-based or microfiche-based mediums. 

FTP — An acronym for of File Transfer Protocol, the protocol (agreed-upon format) used on the Internet for 
sending files. 

Format — The manner in which data, documents, or literature are organized, structured, named and described, 
classified, and arranged. For example, full narrative texts in English language could be in the following forms: 
books or articles, abstracts of text used in reviews and summaries, indexes and catalogs, maps, photographs, 
drawings, sound recordings, video tapes, bibliographies, and statistical and other numeric kinds of tabulations. A 
screen format is the layout of fields on the screen. A report format is the layout of the printed page including print 
columns and page headers and footers. A record format is the layout of fields within a record. A file or database 
format is the layout of fields and records within a data file, layout codes within a word processing document, or 
display lists (vector) or bit maps (raster) within a graphics file. The term is sometimes used to refer to the way 
digital data is encoded or inscribed. Archivists used the terms "genre" or "form of material" to cover what is 
meant by format in this context. 

Frames — A feature supported by most modern Web browsers that enables the Web author to divide the browser 
display area into two or more sections (frames). The contents of each frame are taken from a different Web page. 

GIF — Pronounced jiff or giff (hard g), it is short for graphics interchange format, a bit-mapped graphics. GIF 
supports color and various resolutions. It also includes data compression, making it especially effective for 
scanned photos. 

GILS — Short for Government Information Locator Service. A metadata tool for identifying, locating, and 
describing publicly available Federal information resources, including electronic information resources. 

Gopher — ^A method of making menus of material available over the Internet. Gopher pre-dates the World Wide 
Web for organizing and displaying files on Internet servers. 

Government Electronic Information Service — The system or method by which a component of the 
Government, or its authorized agent, disseminates Government information products to the public via such means 
as a network or use of CD-ROMs at a kiosk. 

Government Information — Refers to information, regardless of form, medium, or format, that is created or 
compiled by employees of a component of the Government, or at Government expense, or as required by law. 
Government information as used here does not include information for official use only, information classified for 
reasons of national security, or information used strictly for administrative or operational purposes (e.g., not of 
public interest or educational value). 

Government Information Product — A Government publication or other work of the United States Government 
conveyed in a tangible physical medium such as a book or CD-ROM, or disseminated through an electronic 
Government information service established and maintained by a Government agency or its authorized agent. 

Graphical User Interface — A program interface that takes advantage of the computer's graphics capabilities to 
make the program easier to use. 

HTML (HyperText Markup Language) — The authoring language used to create documents on the World Wide 
Web. 



HTTP — Short for HyperText Transfer Protocol, the underlying protocol used by the World Wide Web. HTTP 
defines how messages are formatted and transmitted and what actions Web servers and browsers should take in 
response to various commands. 

Information Intermediary — Refers to any person, institution, or mechanism that adds value to information 
products so that they are more useful to information users. Intermediaries perform their work at the middle 
information life cycle stages — that is, between information creation and information disposition or destruction. 
Federal libraries and information centers are examples of intermediaries. 

Information Life Cycle — The various stages through which information passes, including creation, production or 
collection, review and editing, organization and reorganization, packaging, storage, search and retrieval, 
communication and re-communication, dissemination, disposition, archiving, and destruction. 

JPEG — ^Joint Photographic Experts Group (JPEG) is an image compression format used to transfer color 
photographs and images over computer networks. 

Java Applets — The use of small Java programs. Java Applets allows Web pages to include functions such as 
animations, calculators, and other fancy tricks. 

Javascript — A scripting language developed by Netscape to enable Web authors to design interactive sites. 

MARC — Short for machine-reading cataloguing. The USMARC formats are standards for the representation and 
communication of bibliographic and related information in machine-readable form. The Library of Congress, in 
consultation with various user communities, maintains USMARC formats. 

MPEG — Short for Moving Picture Experts Group, a working group of ISO. MPEG generally produces high- 
quality video. 

Medium — The physical, chemical, or biological substrate used to create, organize, store, search for, retrieve, 
disseminate, or permanently archive data, documents, or literature including, for example, paper, microforms, 
fiber optic cables, photographic film, CD-ROM, floppy diskettes, magnetic storage devices, sound recordings, and 
videotape. 

Metadata — Metadata is data about data. Metadata describes how and when and by whom a particular set of data 
was collected and how the data are formatted. These data can be collections or individual instances of objects or 
documents, Internet resources, etc. 

Migration — The transfer of an information product from one hardware type, software package, system, network, 
format, or medium to another. The transfer of an information product from a pre-electronic medium such as paper 
or microform to an electronic medium such as CD-ROM is an example of migration. 

PDF — Short for Portable Document Format. A file format developed by Adobe Systems, PDF captures 
formatting information from a variety of desktop publishing applications, making it possible to send formatted 
documents and have them appear on the recipient's monitor or printer as they were intended. 

Permanent Public Access — The indefinite, continuing accessibility of Government information products by the 
public including the policies, programs, formats, mediums, and standards used. Simply because data such as 
weather and tide information, lists of Government contractors, editions of annual reports, and statistical series are 
continuously updated with more current information does not necessarily mean that provision should not be made 
for retaining (making permanently accessible) the replaced information. Care should be exercised in this regard, 
however, to ensure no harm is done to the public by making available out-of-date information. 

Preferred Medium or Format Standard — A medium or format standard that may not be agency mandated but 
is either common agency practice or applies to a new or promising product format or medium. 



Public Domain — term of American copyright law referring to works that are not copyright protected; free for 
all to use without permission. 

Refreshing — A technical term meaning the manner in which information mediums and technologies are 
periodically reassessed and upgraded to ensure that they are not becoming obsolete, thereby risking the loss of 
information and the compromising of Permanent Public Access policies. The term is also sometimes applied to 
formats and standards (not just mediums) that are in danger of becoming obsolete and need to be replaced or 
upgraded. 

Rich Text Format (RTF) — A standard for specifying formatting of documents. RTF files are actually ASCII 
files with special commands to indicate formatting information, such as fonts and margins. 

SGML — Short for Standard Generalized Markup Language, SGML is a system for organizing and tagging 
elements of a document. SGML was developed and standardized by the International Organization for Standards 
(ISO) in 1986. 

Standard- — An agreed-upon authoritative convention, whether formal or informal, whether official or unofficial, 
whether de facto or de jure, by which information products are created, produced, formatted, published, stored, 
communicated, and moved through the remainder of the stages of the information life cycle. Standards (and the 
more informal concept and term "guidelines") minimize incompatibility and interoperability problems when an 
information sender tries to move data, documents, or literature into and out of several different formats and 
mediums to a receiver. Standards may be set by formal national or international standard-setting bodies, or by 
agencies, or by groups of users informally by common consent. 

TIFF — The acronym for Tagged Image File Format, a graphic file format developed by Aldus and Microsoft. 

Telnet — A terminal emulation program for TCP/IP networks such as the Internet. The Telnet program runs on a 
user’s computer and connects his/her PC to a server on the network. This enables users to control the server and 
communicate with other servers on the network. 

Type of Data — The general nature of the information content of a Government information product and how it is 
arranged, structured, and presented for ease of handling in a medium. Some examples include textual data, 
graphical data, spatial data, numerical data, etc. 

WAIS — Short for Wide Area Information Server. A distributed information service that offers simple natural 
language input, indexed searching for information retrieval, and a relevance feedback mechanism. It has an easy- 
to-use interface that searches all documents relative to your query, ranks them, and makes them available to 
retrieve. 

WAV — The format for storing sound in files developed jointly for Microsoft and IBM. 

WORM — Short for write once, read many, an optical disk technology that allows one to write data onto a disk 
just once. 

World Wide Web (WWW) — WWW is a system of Internet servers that support specially formatted documents. 
The documents are formatted in a language called HTML (HyperText Markup Language) that supports links to 
other documents, as well as graphics, audio, and video files. 

XML — Short for extensible Markup Language. XML is a pared-down version of SGML, designed especially for 
Web documents. It enables designers to create their own customized tags to provide functionality not available 
with HTML. 
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Site Visits to Three Federal Depository Libraries 



Libraries Visited 



McKeldin Library 

University of Maryland 

College Park, Maryland 

(Regional Federal Depository Library) 

Date visited: July 30, 1998 



Washington College of Law Library 
American University 
Washington, D.C. 

(Selective Federal Depository Library) 
Date visited: August 11,1 998 



Rockville Regional Library 
Montgomery County Public Libraries 
Rockville, Maryland 
(Selective Federal Depository Library) 
Date visited: September 9, 1998 



Interviewees 



Linda Spitzer, Acting Head 
Government Documents and Maps 



Joanne Zich, Chief 
Government Documents and Media 
Services 

Patrick Kehoe, Library Director 



Judith Horowitz, Documents Coordinator 
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Interview Questions for Site Visits to Federal Depository Libraries 



Date of Site Visit 

Contact Name 

Title 

Library Name 

Address 



Phone Fax 



Email 



Background Information 

1 . How long have you been in your current position? 

2. Describe range of duties related to Government documents? 

3. How long has this institution been a Federal depository library? 

4. Who are your primary and secondary users of Government information products? 

5. Do you know what mediums patrons tend to be more comfortable using? (e.g., paper, CD- 
ROMs, Internet, diskettes or does it matter?) What about what formats they prefer to use? 
(text and graphics) 



Accessibility for Users 



6. What key issues or concerns do users have about accessing Government electronic 
information products? Have you conducted any user surveys or focus groups to 
determine the needs of patrons who use Government information products and 
services? 
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If they do not mention these issues, probe for them: 

• Bibliographic and findings tools to identify and describe online and electronic Government 
products 

• Problems with changing URLs 

• Charging user fees (specify types of fees: printing costs, fees to access products) 

• Copyright restrictions (can you give an example?) 

• Downloading large files from the Internet 

• User guides and documentation 

• Providing reference and other public services to people with disabilities 

• Other (specify) 



Concerns about Services and Resources 



7. What are your concerns about providing access to electronic Government information 
products? 



If they do not mention, probe for: 

• Are you notified that a product is available through electronic and online storage? If yes, 
how? 



• Are you currently notified that a product has been removed from the Internet? If so, how? 



• Is there a better way to inform the depository libraries of additions, discontinued products, or 
modified products? If so, what are your recommendations? 



• What about retention and preservation of Government electronic information products in the 
future? 



(Hardware and software issues to probe for:) 



• What do you think about the latest minimum recommended specifications for public access 
workstations? 



• Is your library able to keep pace with these minimum specifications? 



• Do the current recommended minimum specifications accommodate current online and 
electronic Government information products? 



• Other hardware/software issues 



Recommendations for Improving Access 



8. Within the last three years, have you or your predecessor conducted any formal or 
informal studies (i.e., surveys, observations) of your users to identify specific needs or 
concerns they have in using Government electronic information products? 



9. If yes, could you please briefly describe what you found and what, if anything, you did 
to make adjustments in your services or resources? 



10. If you were given a pot of money (say $25,000) to improve access to Government 
electronic information products, how would you spend the monies, in priority order? 



11. If you had unlimited resources, what would you do to improve public access to online 
and electronic Government information products? 



12. Are there other issues or concerns you would like to share with us about the transition 
to a more electronic FDLP that we haven*t already covered? 



NCLIS Assessment of Electronic Government Information Products 
Summary Notes for Site Visits to Three Depository Libraries 



1. What mediums do patrons tend to be most comfortable using? (e.g., paper, CD-ROMs, 
Internet, diskettes)? 



• Paper and microfiche; Internet a close second. 

• CD-ROM is the least preferred medium (by librarians and patrons). 



2. What key issues or concerns do users have about accessing Government electronic 
information products? 



Accessing Electronic Resources 

All three librarians expressed major concerns about the problems in using Government- 
produced CD-ROMs that are not standardized: 

• The search and retrieval software is different for each CD, often they come with no 
installation instructions or user documentation, and they are not user-friendly. 

• Librarians must call in a computer technician to load the CDs and show librarians or 
users how to access the information. 

• Two of the three libraries have significantly reduced the number of CD-ROM titles they 
select because of the above-mentioned problem. The third librarian indicated that they 
have a ton of CD-ROMs that are not used due to the problems with loading, accessing, 
and using them. 

Users are still intimidated by electronic mediums and computers. Most users ask librarians 
to help them search for materials on the web. 

Since most Government websites only contain the most recent information, all librarians 
expressed concerns about users accessing retrospective Government information on the 
web. Two selective depository librarians often send their patrons to the regional depository 
to use their paper files for some historical Government information. 



3. What key library issues or concerns do you have about providing access to electronic 
Government information products? 



Charging Fees 

Two of the three libraries do not currently charge fees for printing materials from the 
Internet or CD-ROMs. One library was planning to change its policy beginning in 
September 1998 because it cannot continue to fund this expense. The second library is also 
contemplating changing its policy after installing the next generation computer system. 





Hardware issues 



• Patrons expect the library to have state-of-the-art equipment and expect librarians to 
know how to use it. Although all librarians felt their libraries had state-of-the-art 
equipment, they were concerned about how they could meet rising user expectations for 
the “best” computer hardware. 

• Users do not have access to enough workstations, so the libraries must impose a time 
limit on computer use. 

• Costs of hardware, telecommunications, and access to electronic information have 
increased dramatically. For example, one library reported spending $60,000 on these 
services and equipment 10 years ago; now they spend about $270,000 on hardware 
including maintenance agreements. 

Training Issues 

AH librarians expressed concerns about finding time and money to train librarians and 
staff, especially on the use of CD-ROM products, but also on downloading files, effectively 
searching the Internet for Government information, and creating and maintaining web 
pages. 



4. If you were given a pot of money (e.g., $25,000) to improve access to Government electronic 
information products, how would you spend the monies, in priority order? 



• New CD-ROM server; an 18-disk CD changer (3 responses). 

• More staff training (2 responses). 



5. If you had unlimited resources, what would you do to improve public access to online and 
electronic Government information products? 



• Form partnerships with GPO and a Government agency to put some retrospective 
Government information on their server so it will be accessible to users in the future (2 
responses). 

• Provide outreach to public schools, community centers, etc., to educate students and 
adults about the wide variety of valuable information available from the Federal 
Government (2 responses). 

Other Comments: 

One librarian expressed strong feelings about the need for Congress to provide long-term 
flnancial support to Federal depository libraries so they can provide permanent public 
access to digital media. 
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Electronic Government Information Products Assessment 
Agency Meetings Held 



Agency 



Number of Attendees 



Department of Health and Human Services 1 0 

Meeting Date: September 14, 1998 

Supreme Court of the United States 1 1 

Meeting Date: September 15, 1998 

Environmental Protection Agency 1 2 

Meeting Date: September 17, 1998 

Department of Education 5 

Meeting Date: September 22, 1998 

Department of Commerce 8 

Meeting Date: September 23, 1998 



National Archives and Records Administration 
Meeting Date: September 24, 1998 
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Electronic Government Information Products Assessment 
Agency Meeting Discussion Questions 



1 . Does your agency have preferred medium and format standards for Government electronic 
information products at the permanent public accessibility stage? If so, what are the top 
three? What factors does your agency consider in determining preferred standards (e.g., user 
needs, agency's dissemination requirements or policies, cost, security etc.)? What about 
specific standards for CD-ROMs as they relate to user documentation, installation, search 
software, etc.? 

2. Can you give us any examples of particularly innovative and creative product formats, 
mediums and/or online approaches? We have in mind formats, mediums, or online 
approaches that may well point to the wave of the future, not only for a particular product but 
also for other kinds of products, yet is neither an agency-mandated standard nor even a 
common agency practice. 

3. Is there any difference between your agency's preferences for mediums and formats as 
opposed to the preferences of intermediary distributors? If so, what are those differences, and 
why are the two preferences different? 

4. Has your agency involved external user groups in assessing the value and effectiveness of the 
dissemination of Government electronic information products? If so, are there formats and 
mediums that seem particularly appropriate for public dissemination to users who may be 
economically, technically, or physically disadvantaged? 

5. Does your agency follow any internally or externally prescribed guidelines for the 
presentation and organization of products in online formats? If so, what are they (e.g., 
WWW Federal Consortium, FIPS Guidelines, agency or departmental publication 
specifications or guidelines)? 

6. Has your agency undertaken any kind of cost benefit analyses for producing or creating 
products in preferred or emerging formats, mediums, or online approaches for distribution to 
the FDLP? If so, which ones appear to be the most cost-effective? 

7. What factors does your agency consider in deciding to create or retain products in more than 
one medium? Is this a common agency practice? 
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8. Are there trends with respect to migrating specific families of products from pre-electronic 
mediums to electronic mediums or formats? For example, are loose-leaf publications, 
training manuals, annual reports, conference proceedings, newsletters, rules and regulations, 
scientific journals, etc., targeted for migration to a particular medium? If so, which mediums 
and formats are used for specific families of products? 

9. Has your agency identified any medium and format standards that seem particularly 
appropriate for use throughout a product’s entire information life cycle, not just at one stage 
(i.e., creation, storage and retrieval, communication and dissemination, archiving and 
disposition) for electronic Government information products? If so, which ones? 

10. How do you determine whether a product should be made permanently publicly accessible 
when you create or produce it? If so, what criteria do you use to determine which products 
will be permanently publicly accessible? Can you give us any examples of how you ensure 
permanent public accessibility for a given product? 

11. Does your agency routinely provide locator tools (e.g., GILS or specific agency locators) to 
enhance access to information sources and services available to external users and customers? 
If so, is this an official policy, common agency practice, or both? 

12. Are there trends for facilitating public access to your agency products by including them in 
broad electronic Federal Government information services such as GPO Access, LOG 
Thomas, and NTIS FedWorld? Are you using any particular guidelines to facilitate that 
decision, and if so, what are they? 



BEST COPY AVAILABLE 
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Summary of Responses to Agency Meeting Questions 



1. Does your agency have preferred medium and format standards for Government electronic 
information products at the permanent public accessibility stage? If so, what are the top 
three? What factors does your agency consider in determining preferred standards. 



Agencies reported using the following electronic mediums most often: CD-ROM, Internet, 
and Bulletin Board System (BBS). The most frequently used formats include: 

• HTML, PDF, ASCn 

• TIFF, GIF, JPEG 

• Lotus/Domino 

In determining medium and format standards, agencies consider the amount of information 
or files used, the timeliness of the information (e.g., more recent products or publications 
often placed on the web), and user needs for easy and quick access to information. For 
example, some agencies are looking to Windows as an interface for CD-ROM products 
since the public is used to seeing and using Windows. 



2. Do you have any examples of particularly innovative and creative product formats, mediums, 
and/or online approaches? We have in mind formats, mediums or online approaches that may 
well point to the wave of the future for not only a particular product but also for other kinds 
of products, yet is neither an agency-mandated standard nor even a common agency practice. 



Almost all the agencies interviewed are exploring a wide range of innovative and creative 

format, medium, and web applications. Below is a sampling of some of the interesting 

online approaches and formats used by the agencies interviewed. 

• Data warehousing: Taking information not previously publicly accessible and 
integrating it into an online format. The format used is an Oracle database using SQL 
to query. 

• Interactive Geographic Information System (GIS). Provide mapping capability through 
GIS combined with regulatory information to create dynamic maps. 

• Online catalog of all products on the agency’s website, using Oracle with a ColdFusion 
interface with their search engine (Verity). 

• Creating user guides for CD-ROMs as pop-up HELP or short Read-Me files so users 
will be more inclined to use HELP. 

• Live “real-time” web casting of selected speeches. Format: RealPlayer software, 
available free from the web. 

• Radio news broadcast news service. Provides daily radio sound-bits for news reports. 
Format: RealAudio RealPlayer software and WAV file format for downloading. 
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• Real-time forecasting of air pollution levels for 22 states from one site. Format: 
animated GIFs created by nonproprietary software designed by computer center in 
North Carolina. 

• Multimedia CD-ROM (i.e., art, music, animation, film, and video). 

• Searchable electronic inventory of all proposals funded over the last 30 years so the 
agency can analyze its own information and make it available to others. Format: legacy 
database put into WAV database using HTML on the fly. 

• Documents stored in TIFF format for image and textual data. As customers request 
documents, the agency converts them to PDF so customer can download. 



3. Is there any difference between your agency's preferences for mediums and formats as 
opposed to the preferences of intermediary distributors? 



Generally, agencies indicated that intermediary distributors do not find agency formats to 
be restrictive. However, the distributors often modify formats (e.g., from HTML to ASCH, 
or reformat data using compression technology). 



4. Has your agency involved external user groups in assessing the value and effectiveness of the 
dissemination of electronic Government information products? If so, are there formats and 
mediums that seem particularly appropriate for public dissemination to users who may be 
economically, technically, or physically disadvantaged? 



Yes, all agencies reported that they involve users in assessing some aspects of their 
products, per OMB Circular A-130. Examples include the following: 

Focus groups are used to: 

• Determine the capability of new products (e.g.. Can you use the same technology for 
the newer version of a product? Do users lose anything (e.g.. Macros) when they 
update a product?). 

• Determine ways to create more user-friendly CD-ROMs that resulted in the agency 
establishing three principles for producing CD-ROMs: make them simple to use, 
intuitive, and self-tutorial. 

• Determine how information is presented on the web (Alpha and Beta testing). 

• Solicit feedback on usability and accessibility; focus groups conducted with tribal 
Governments, teachers, librarians, children, etc. 

• Learn about expectations, behavior, and problems in accessing products on the web 
experienced by the elderly. 

• Interview and videotape users to assist agency redesign of website. 
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• Solicit ideas from educators at professional conferences about topics that they would 
like to see presented in online products. 

For technologically or economically disadvantaged customers, one agency sub-unit faxes 
free copies of information printed from the CD-ROM or the Internet. Another agency sub- 
unit reported they try to reach economically disadvantaged customers by training rural 
co mm unity leaders in isolated areas on ways to access health-related information on the 
web . 



5. Does your agency follow any internally or externally prescribed guidelines for the 
presentation and organization of products in online formats? If so, what are they (e.g., 
WWW Federal Consortium, FIPS Guidelines, agency or departmental publication 
specifications or guidelines)? 



Most agencies have developed guidelines or “best practices” for presentation of products in 
online formats. However, several agency representatives indicated that the real challenge is 
in convincing agency staff to comply with the guidelines. The following agencies provided 
information on their guidelines: 

1. NARA: NARA Guidelines for Digitizing Archival Materials for Electronic Access (not 
to be considered a standard for digital imaging). 

(http ://www. nara. gov/nara/v ision/ eap/eapspec . html) 

2. Federal Web Consortium’s guidelines are based on Dept, of Education’s guidelines 
(http://www.ed.gov/internal/wwwstds.html). One sub-unit, NCES, also has developed 
guidelines. 

3. EPA used WWW Federal Consortium guidelines 
(http://www.dtic.mil/staff/cthomps/guidelines/) to develop their own guidelines for 
presentation. 

4. Census uses a process and structure for submitting items for the web, but it is not yet 
formalized. 

5. No departmental guidelines exist for the fifth agency, but most sub-units have some 
land of guidelines for presentation and organization, although they might vary among 
sub-units. 
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6. Has your agency undertaken any kind of cost-benefit analyses for producing or creating 
products in preferred or emerging formats, mediums, or online approaches for distribution to 
the FDLP? If so, which ones appear to be the most cost-effective? 



Generally, agencies have not conducted a formal cost-benefit analysis. Agency 

representatives made the following observations, however: 

• One agency sub-unit tracks the number of customers who purchase a product and 
compares this amount against the cost of producing it. They discontinued a product on 
CD-ROM because so few people could afford to buy it. 

• The web reduces administrative costs for printing and mailing hard copies of 
publications. 

• One agency reported a dramatic decrease (by 25,000) in the number of publications 
requested under the Freedom of Information Act due to the web. 

• One agency tracks the number of people that a product can potentially reach (e.g., they 
sell 2,000 paper copies through GPO, but have 9,000 hits on the website). 

• One agency reported that they order fewer publications to fill customer requests as a 
result of the web. 



7. What factors does your agency consider in deciding to create or retain products in more than 
one medium? Is this a common agency practice? 



Agency representatives reported that they consider a variety of factors in creating and 
retaining products in more than one medium, although they did not characterize these 
factors as a common agency practice. Several agencies reported that these issues are 
considered on a casc-by-case basis or by the individual program unit, 
considered are: 



me Key lactors 



1. Budget (e.g., some products in CD-ROM are too expensive to make available to a small 
audience). 

2. Cost (e.g., cost to print and mail product as opposed to make it available on the web). 

3. Needs of technologically disadvantaged users (e.g., one agency maintains its Fax on 
Demand service, even though it is not cost-effective). 

4. Accessibility (e.g., one sub-unit stores products in TIFF image format so they can 
produce them in whatever medium of output customers want). 

5. Size of audience (e.g., agencies survey users and use web software to track use). 

6. Number of queries or type of customer requests. Customers will often request a 
publication or product in more than one medium. 
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7. Regulatory requirements. Some products must be produced in paper regardless of 
whatever other medium it is produced in. 

8. Type of product/publication (e.g., searchable databases are only suitable for electronic 
mediums). 



8. Are there trends with respect to migrating specific families of products (e.g., loose-leaf 
publications, training manuals, annual reports, conference proceedings) from pre-electronic 
mediums to electronic mediums or formats? If so, which mediums and formats are used for 
specific families of products? 



The general trend reported by agency representatives is to migrate more products to the 
web, especially recent ones. Some examples include: 

• Conference proceedings and presentations online in PowerPoint or PDF. 

• Newsletters in HTML. 

• Training manuals and annual reports in HTML and PDF. 

e Information for records managers are posted to Gopher, but will move to the 
agency’s website in 1999. 



9. Has your agency identified any medium and format standards that seem particularly 
appropriate for use throughout a product’s entire information life cycle, not just at one stage 
(i.e., creation, storage and retrieval, communication and dissemination, archiving and 
disposition) for electronic Government information products? If so, which ones? 



Most agencies have either not addressed this issue of information life cycle or are struggling 
with it. 

• NARA has established medium and format standards for transferring permanent 
records to the National Archives in 36CFR 1228.188. 

• One sub-unit reporting putting documents in ASCH, but using Oracle for database 
management. 

• One sub-unit is beginning to think about standardization for some documents. They 
draft documents in Lotus Notes (GroupWare) and publish final document in 
another database that goes onto the web. They use Rich Text Format (RTF) to 
accommodate images and text. 

• Several sub-units indicated that the technology is changing so rapidly they cannot 
establish standards. 
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10. How do you determine whether a product should be made permanently publicly accessible 
when you create or produce it? If so, what criteria do you use to determine which products 
will be permanently publicly accessible? Can you give us any examples of how you ensure 
permanent public accessibility for a given product? 



No agencies could provide responses to this question or indicated that this issue has not 
been resolved. Some observations: 

• One sub-unit is committed to making paper and CD-ROM-based products available 
for permanent access, but they are less clear about their commitment to products on 
the Internet. 

® Some agency representatives did not understand the differences between permanent 
public access and permanent records. 

(The experts interviewed for this study provide some insight into the reasons that agencies 
are not addressing this issue.) 



11. Does your agency routinely provide locator tools (e.g., GILS or specific agency locators) to 
enhance access to information sources and services available to external users and customers? 
If so, is this an official policy, common agency practice, or both? 



Most agencies indicated that the web format supercedes the original GILS concept. 
However, most agencies have their own locators: 

• NTIS has a catalog and maintains some GILS records. 

• EPA’s website has a GILS record and they put all Internet products on one server so 
there is one access point for all their products. 

® Development and maintenance of GILS records is official agency policy for NARA. 



12. Are there trends for facilitating public access to your agency products by including them in 
broad electronic Federal Government information services such as GPO Access, LOG 
Thomas, and NTIS FedWorld? Are you using any particular guidelines to facilitate that 
decision, and, if so, what are they? 



About half of the agencies use GPO Access or NTIS FedWorld. The other half relies more 
heavily on individual agency websites with good links. 
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Assessment of Electronic Government Information Products 
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Assessment of Electronic Government Information Products 
List of Expert Interviews and Discussion Questions 



Interviewees 


Date of Telephone Interview 


Jerry Malitz, Webmaster 
National Center for Education Statistics 
U.S. Department of Education 
Washington, D.C. 


October 27, 1998 


Linda Wallace, Chief 
Electronic Information Services 
Internal Revenue Service 
Washington, D.C. 


October 27, 1 998 


Evelyn Frangakis, Preservation Officer 
National Agricultural Library 
U.S. Department of Agriculture 
Beltsville, Maryland 


November 10, 1998 


Abby Smith, Director of Programs 
Council on Library and Information Resources 
Washington, D.C. 


November 10, 1998 


John Bertot, Associate Professor 
State University of New York at Albany 
Albany, New York 


November 18, 1998 


Charles McClure, Distinguished Professor, 
School of Information Studies 
Syracuse University 
Syracuse, New York 


November 24, 1998 



Interview Questions for Webmasters: 
Jerry Malitz and Linda Wallace 
(October 27,1998) 



Role of Webmaster 

1 . How long have you been in your current position as webmaster? When and how was the position 
created? Were you the first webmaster in your agency? How does the position reside 
administratively in the structure of your agency? What office or unit do you report to? 

2. Please describe your current job responsibilities and duties. What portion of the following skills, 
experience, and knowledge do you use to perform your job: technical, administrative, analytical, 
program, other? 

3. Is there a formal or informal structure for working with staff and administrators in other departments 
or units (e.g., program staff, IT, publications, public relations, records managers, librarians, etc.)? If 
yes, please describe how you interact with them? 

4. What do you envision as the future role of the webmaster in Federal Government agencies? Do you 
see your role as being very different in 5 years than it is now? How? 



Format Standards and Public Accessibility 

5. Please describe the website development process in your agency from the time you receive or 
generate requests through design, development, evaluation, testing, and implementation, etc. 

6. Has your agency developed policies or guidelines including format standards to ensure technical 
consistency in the development of web products that are intended for public dissemination? What are 
the most frequently used file formats and why? Can you identify any formats you plan to use in the 
ftjture? 



7. Are there limitations or specific designations of software tools that may be used to develop and 
implement web pages or sites? What standards are applied to configuration control and arrangement 
of web-based applications? Do you have a direct role in determining these standards, or are they 
developed at an agency or departmental level? 

8. Are there general security standards applied to the availability or distribution of web-based 
information? What is your role in determining or implementing these standards? Which software 
products do you use to implement these standards? 
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9. How do you evaluate the effectiveness of your websites? Who is involved in the process? What 
methodology and criteria have you or others used to evaluate websites? 



10. Has your agency discussed the concept of permanent public accessibility as it relates to Government 
electronic information products intended for public dissemination? How is your agency addressing 
the concerns of librarians, GPO, and others for ensuring permanent public accessibility for electronic 
Government information products? 



11. What consideration are you giving to creating a metadata record for your information resources or 
services on the web (e.g., GILS, MARC, or specific agency locators)? 



Cost Analysis 

12. Linda, we know you have collected data on the comparative costs of delivering services to customers 
via different delivery mechanisms such as mail, e-mail. Fax on Demand, kiosks, Internet, telephone, 
walk-in, CD-ROM, etc. What have you learned about the costs of delivering services to customers 
using these different systems? Which delivery mechanisms are the most cost-effective for what types 
of services? 



NCLIS Assessment of Electronic Government Information Products 



Summary Notes from Interview with Linda Wallace (IRS) and Jerry Malitz (NCES) 



1 . Please describe your current job responsibilities and duties. 



WALLACE (IRS): 

(Wallace was a telecommunications expert and technical advisor to CIO when IRS asked her to be 
webmaster. She also holds the title of Chief, Electronic Information Services. She is responsible for 
all electronic information products including Fax on Demand, Internet, e-mail, etc.) 

Wallace’s three major areas are content, applications, and development. Her office: 

• Generates new services and authorware, including creating automated filters and templates 
through core knowledge repository. 

• Participated in the development of SGML format (standard format for IRS since 1970s). 

• Interacts with customers to automate a standard way to build a core knowledge repository. 
Repository contains automated templates and filters to generate media output to serve 
customers via Internet, Fax on Demand, CD-ROM, bulletin board system, telephone, or mail 
requests. 

a. Core repository can satisfy 95 percent of requests using 86 different variables or 
attributes that are indexed so everything is searchable. All documents include individual 
catalog and document numbers. 

b. Documents are always authored in SGML and can automatically be converted into a 
different format or posted on the web, BBS, etc., in 10-12 hours to fill customer 
requests. 

c. Filters and templates are solely by this group. They also track history of a document. 

d. A knowledge base is being built by developing a database of frequently asked questions. 

e. They use ICON tagging to provide accessibility to the visually impaired. All IRS 
documents are ADA-compliant, online searchable, and downloadable. 

MALITZ (NCES): 

As Technology Outreach Officer for NCES, Malitz services state education agencies, school districts, etc. 

a. Each one of NCES's 30 programs has a web publisher and a web liaison. 

b. Contractors actually prepare materials for the web once program officer has approved content. 

c. Malitz sets standards, guidelines, and procedures for web publisher to follow. 
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d. Web publisher in program area develops website on a separate development server; Malitz 
reviews and makes technical changes to ensure that the site meets minimum standards and 
guidelines. 

e. Sometimes, he develops a new application for others to use (e.g., NEWS FLASH subscription 
service featuring daily breaking news from the Department of Education). 



2. Do you have a formal or informal structure for working with staff and administrators in other 
departments or units (e.g., program staff, IT, publications, public relations, records managers, 
librarians, etc.)? If yes, please describe how you interact with them. 



WALLACE: 

Wallace deals with high-level senior executives, reviewing their business plans, problems, and goals, 

then recommending solutions that include productivity measures, production rates, and cost per 

person. In one case, she recommended a business CD-ROM. 

• Establishes strong liaisons with industry (has marketing person on staff). They receive one-half 
of funding from industry to support business projects that benefit industry and IRS customers. 

• Established various delivery service programs: Internet in ’96, Fax on Demand in '96, CD- 
ROM in '95, BBS a while ago. 

MALITZ 

® Each one of NCES’s 30 programs has a web publisher and a web liaison. 

• Contractors prepare materials for the web once content has been approved by the program 
officer. 

• Web publisher in program area develops web site on a separate development server; when 
finished, Malitz reviews and makes technical changes to ensure that site meets minimum 
standards and guidelines. 

• Sometimes, Malitz develops a new application for others to use such as the NEWS FLASH 
subscription service that features daily breaking news from the Department of Education. 

• Malitz’s work is divided fairly evenly between technical, administrative, analytical, and 
program areas. 



3. What do you envision as the future role of the webmaster in Federal Government agencies? How you 
see your role as in 5 years as compared to now? 



MALITZ: 

The role of webmaster will be completely different. In the future, he/she will have more of a 
coordinating function and will set policies and procedures. The program staff will be forced to do 
their own work on the web, just as they now do their own word processing and e-mail. 

WALLACE: 

The role of the webmaster will be that of an enabler for business units with everyone involved. 
There will be more of a focus on multimedia (e.g., BBS, CD-ROM, Fax on Demand) and not just the 
Internet. 



4. Please describe the website development process in your agency from the time you receive or 
generate requests through design, development, evaluation, testing, and implementation, etc. 



WALLACE: 

Her unit receives and generates requests. Requests from the core repository can fit into an existing 
template filter application. 

a. Staff and contractors conduct testing and implementation. 

b. Second year after web in operation, requests for paper copies of forms dropped by 50 
percent. 

c. Provide hidden codes to track where returns come from: fax, Internet, libraries, phone 
requests, etc. 

d. Evaluation includes a simple three-question customer service survey on content: did you get 
what you needed, where would you have gone if not here? They build evaluation into every 
step of the process. 

e. A panel of experts measures the effectiveness of their websites. Also, they have partnered 
with schools to recruit instructors and students to review site before they go "live." 

f. One person reviews all e-mail messages that contain feedback on website; use automated 
sorters by key words to batch the type of feedback received. 



MALITZ: 



Before website, customers were very specialized. Most were data fde users. After they created their 
website, their customer based increased tremendously. Now the culture is different and NCES is 
dealing with questions from the general public. 

a. NCES uses a developmental server but is planning to implement a Point-to-Point Tunnel 
Protocol (PPTP) so the developmental server is behind the firewall and no longer open to 
everyone. Only web publishers and contractors will have access to server. 

b. Malitz never reviews content; that is done by individual program staff. 

c. NCES conducts customer surveys of users to develop and refine sites. 

d. Malitz does database development and tests multiple browsers. NCES is UNIX-based; rest 
of ED is Windows-based. 



5. Has your agency developed policies or guidelines including format standards to ensure technical 
consistency in the development of web products that are intended for public dissemination? What are 
the most frequently used file formats and why? Can you identify any formats you plan to use in the 
future? 



WALLACE: 

Formats most frequently used are SGML, PDF, HTML, and Postscript, respectively. They will add 
XML soon. They train authors to use SGML. SGML is “intelligent” data that can automatically 
generate other formats. Most agencies do not use SGML because it is harder to author in. 

Wallace’s agency uses it because it is much more robust, and it is easy to change a document format 
to match customer needs (e.g., tax law information for consumer and for lawyers). 

MALITZ: 

NCES uses PDF, then HTML (optional). They rarely put entire publication in HTML. 



6. Are there limitations or specific designations of software tools that may be used to develop and 

implement web pages or sites? What standards are applied to configuration control and arrangement 
of web-based applications? Do you have a direct role in determining these standards, or are they 
developed at an agency or departmental level? 



WALLACE: 

Their focus is knowledge-based, not web application. Her department sets the standards. They use 
C++ and Perl. 



MALITZ: 



All publications are in PDF; all else in HTML. They use SQL databases to support the web. Malitz 
has a direct role in determining standards. 



7. Are there general security standards applied to the availability or distribution of web-based 

information? What is your role in determining or implementing these standards? Which software 
products do you use to implement these standards? 



WALLACE: 

The IRS uses an automated redacting scheme; with one keystroke, they can create a public and 
specialized version of the same document. They apply all security standards from the Government, 
including SSA, Treasury, etc. They cannot reveal security software. 

MALITZ: 

Only a few people who use developmental server have access; all must be registered users. NCES 
uses PPTP encryption for the developmental server. 



8. Has your agency discussed the concept of permanent public accessibility to electronic Government 
information products intended for public dissemination? How is your agency addressing the concerns 
of librarians, GPO, and others for ensuring permanent public accessibility for electronic Government 
information products? 



WALLACE: 

All tax forms, instructions, publications etc., are available for 5-6 years online. The core knowledge 
repository maintains material for 14 years, but they do not keep every application back that far. 
IRS can fill e-mail requests for information or forms from earlier years. In addition, they provide 
GOLD CARD SERVICES for librarians. Librarians have their own page, track orders, and talk 
"live" with one another. IRS gives their orders priority. 

MALITZ: 

The issue of permanent public accessibility is currently under discussion. 



9. What consideration are you giving to creating a metadata record for your information resources or 
services on the web? (e.g., GILS, MARC or specific agency locators)? 



WALLACE: 

GILS records are a subset of the 86 variables that go into the core knowledge repository. 
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MALITZ: 



The Dept, of Education has an agency locator with total search capability. They also participate in 
FedStats, White House Briefing Room, etc. 



10. Linda, we know you have collected data on the comparative costs of delivering services to customers 
via different delivery mechanisms such as mail, e-mail. Fax on Demand, kiosks, Internet, telephone, 
walk-in, CD-ROM, etc. What have you learned about the costs of delivering services to customers ’ 
using these different systems? 



WALLACE: 

Breakdown of comparative costs follows: 

a. It costs IRS $3 per call for the public to call into their toll-free number and for IRS to fill the 
request. The cost to IRS for the public to use the Internet to access and use forms is 1 cent, a 
difference of 300 to 1. 

b. The costs to create forms on Internet have gone down, but the cost to fill phone requests 
remains the same. 

c. It costs ms $2.50 to make and distribute to public libraries each CD-ROM containing 5 years 
of tax forms, instructions, and publications. This is less than it takes for mS to respond to one 
telephone call. The mS also sends tax CD-ROMs to the depository libraries. They can mount 
them on their PCs or allow customers to check them out. 

d. They found that kiosks are very expensive; ATMs are cost-effective 

Final Comments from Wallace 

The answer to public accessibility is not the Internet; it is multimedia. Delivery mechanisms must 

meet the individual needs of the customers; no one size fits all. 



Interview Questions for Preservation Specialists: 
Evelyn Frangakis and Abby Smith 
(November 10, 1998) 



1. How long have you been in your current positions? Please describe your current job responsibilities 
and duties. What portion of the following skills, experience, and knowledge do you use to perform 
your job: technical, administrative, analytical, and other? 

2. What are the key problems associated with digital preservation? 

3. What key policy, organizational, economic and other non-technical issues need to be addressed or 
solved to facilitate digital preservation? 

4. What technological strategies or models have various organizations such as the Association of 
Research Libraries, the Digital Library Federation, National Archives, etc., identified to address these 
problems? Evelyn, one of the NCLIS staff mentioned that NAL has established a structure or 
framework that addresses this problem. Could you please talk more about that? If you have any 
handouts you can fax to us, that would also be helpfiil. Abby, can you describe some of CILR's 
recent efforts to address the issue of digital preservation, including the survey by Jeff Rothenberg of 
the RAND Corporation? 

5. What do we know about specific file formats or mediums that might facilitate digital preservation 
such as SGML, CD-ROM, etc.? 

6. Are there any important preservation issues that we have not addressed in the above-listed questions? 
If so, please discuss them. 

7. Could you please refer us to any important articles on this topic that have been published in the last 
year? 
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Summary of Notes from Conference Call with Two Preservation Specialists: 

Evelyn Frangakis (NAL) and 

Abby Smith (Council on Library and Information Resources) 



1. How long have you been in your current position? Please describe your current job responsibilities 
and duties. What portion of the following skills, experience, and knowledge do you use to perform 
your job: technical, administrative, analytical, and other? 



ABBY SMITH (CLIR) 



• Been with CLIR since Sept. 1997 as director of programs. 

• Provide program coordination among the four areas: economics of information, leadership in 
libraries and archives, digital libraries, preservation and access. 

• Her primary program responsibility is in preservation and access in libraries, traditional and 
digital. 

• 10 staff members; 6 professionals, 4 admin, support staff. 

• Spends 75 percent of time on policy-related issues and the remaining 25 percent of time spent 
on administrative functions ( i.e., coordinating publications program) 

EVELYN FRANGAKIS (NAL) 



• Been in current position since January 1997. She is NAL's first preservation officer. 

• Duties: plan, direct, and implement agency-wide programs for ensuring permanent and future 
accessibility of the foremost national collection of materials in agriculture. 



Coordinates activities with other national efforts, such as the U.S. Agricultural Information 
Network (USAIN). Established in 1988, USAIN provides a forum for discussion of agricultural 
issues, takes a leadership role in the formation of a national information policy as related to 
agriculture, makes recommendations to the National Agricultural Library on agricultural 
information matters, and promotes cooperation and communication among its members. NAL 
participates in implementing USAIN’s preservation plan for print materials, A National 
Preservation Program for Agricultural Literature. The USAEV Preservation Steering Committee, 
on which Frangakis serves, oversees this national cooperative plan. Under the auspices of 
Cornell University, the USAIN plan has received two NEH grants to microfilm core national 
and state agricultural literature. To date, 15 states are participating in these grants. Other 
components of the national plan and program include determining what are the important 
archival and manuscript collections of agricultural materials and what approaches can be used 
for their preservation. 



• NAL efforts include developing their own preservation program that includes a traditional 
preservation program and digital efforts. 



• Digital efforts are two-pronged: conversion of brittle paper materials into digital products by 
working with best available guidelines to implement good preservation practices (this digital 
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material will be available on the web); develop a program to preserve USDA digital materials 
(i.e., materials that are born digitally). 

• Helps develop preservation policies and analyze other policies that come to NAL or USDA that 
affect preservation of the collection. 

• Time spent on different types of work at different times. Duties fairly split among policy, 
technical, administrative, analytical areas. 

• Staff consists of two assistants at present. However, NAL leverages its preservation resources 
by establishing cooperative inter-institutional agreements and contributing funds to sister 
institutions in order to further develop the USAIN preservation program (e.g., cooperative 
agreements with Cornell University to establish copyright clearance for core historical 
literature, developing NEH grant proposals). 



2. What is the distinction between digital preservation and permanent public accessibility of electronic 
records (as it relates to, for example, the Federal Depository Library Program)? How long is "long- 
term" preservation vs. "permanent public accessibility?" 



FRANGAKIS 

• Some background from the USDA perspective: The USDA Digital Publications Preservation 
Steering Committee was established this past summer to oversee the implementation of the 
plan, A Framework for the Preservation of and Permanent Public Access to USDA Digital 
Publications. This group met for the first time in October 1998. There was a discussion of 
definitions in order to place into context the universe of material covered by XYieFramework. 
Publication was defined as “a data or information product prepared by the USDA in digital 
form intended to be disseminated to the public.” The Framework defines preservation as “the 
act of permanently maintaining and making available data or information, with all original 
content intact.” 

• Other experts, such as Don Waters of the Digital Library Federation, talk about preserving 
integrity and ensuring persistence of digital information. The Commission’s SGML report talks 
about preservation goals such as enhancing the long-term preservation of and access to 
information of enduring value for as long into the future as possible. 

• THE CPA Digital Archiving Task Force was charged to investigate the means of ensuring 
“continued access indefinitely into the future of records stored in digital electronic form.” 

• Concept of preservation in traditional preservation world examines the concept of permanence, 
but in the print world the concept of permanence relates to chemical inertness and mechanical 
durability. These concepts do not translate easily into a digital world. In the digital world, we 
are no longer dependent on original copies (i.e., original copies do not have the same meaning.). 

• Within NAL, they use digital preservation fairly loosely to speak about both digital efforts: 
conversion of brittle materials and USDA digital publication preservation efforts. Not sure 
professionals in the library and preservation community have a common understanding of what 
it means, even though it is important to come to a common understanding. GPO defines 
permanent access as “Government information products within the scope of the Federal 



Depository Library Program that remain available for continuous no-fee public access through 
the program. The 1996 GPO report to Congress, to Identify Measures Necessary for a 
Successful Transition to a More Electronic Federal Depository Library Program, states that 
“‘preservation’ means that offlcial records of the Federal Government, including Government 
information products made available through the FDLP, which have been determined to have 
sufflcient historical or other value to warrant being held and maintained in trust for future 
generations of Americans, are retained by the National Archives and Records Administration 
(NARA).” 

• At NAL, the mission of its preservation program is "to preserve and ensure access to the 
intellectual content and physical composition of agricultural works of national and 
international importance indefinitely into the future." No timeframe is mentioned because no 
one at this time can say how long into the future information will be needed. 



SMITH 

• There is no standard accepted method of ensuring long-term access to digital information. She 
described preservation goals as permanent or persistent or perhaps more accurate to say that 
one of the primary goals of preservation is to set up systems that "sustain predictable levels of 
loss." 

• Difference between preservation in a digital world and in an analog world is that in a digital 
world, information is completely independent from the medium on which it is carried. In an 
analog world, people try to preserve the media in which information is recorded. No analogy in 
the digital world. No concept of preserving the artifact as an artifact that has its own level of 
information. 

• Problem in digital preservation is that there is no way to ensure that digitally stored 
information can move from one software-hardware configuration onto another through 
generations. Two problems: 

(1) Problem of instability of media in which information is stored (don't know how long CDs or 
other media will last) 

(2) More serious issue from CLIR point of view is that software/hardware configurations on 
which information is stored become obsolete so quickly that even when you migrate 
information from one system to another, much of the information is lost (data and 
functionality). 

• CLIR tries not to talk about "digital preservation" but they cannot avoid it. 

• Other countries may view preservation differently. England interested in American concept of 
digital archiving -preserving the integrity of data; that is, information is original and authentic 
and it can be proved that data have not been changed. According to Smith, scientists say that 
we will solve the problem of authentication, but hasn't been solved thus far. 
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Permanent Public Accessibility Issues (FRANGAKIS) 

• Federal agencies relying on FDLP to serve an "archiving" function for retrospective materials. 

• Question of how to preserve digital information indefinitely into the future has not been 
answered. CLIR and NAL are discussing strategies by opening up dialogue and promoting 
research in this area. 

• Digital Archiving Task Force Report discussed ensuring the integrity and long-term availability 
of digital information through migration. Information on CD-ROM and other media is in a 
format that may or may not be readable into the future due to hardware/software obsolescence. 
Even if media could be preserved, no guarantee that it would be accessible and functional 
indefinitely into the future. No answers anytime soon. 



3. What are the key problems associated with digital preservation? 



SMITH 

• Additional problems: fragility of media and platform dependence issue 

(1) Two additional issues: difficulty of understanding what we can and cannot do under 
current copyright law. Latest iteration of copyright law clarifies copyrighting for 
preservation purposes, but still unclear for access purposes. Library of Congress is 
currently studying this. Copyright law may not have any implications for Government 
information but many vendors create derivatives of Government information and copyright 
it. Government should never be in the position of depending upon the private sector to 
preserve some of this information. 

(2) Any transmission link is as strong as the weakest link. The weak link in the transmission of 
electronic information is not technology; it's human beings. Human infrastructure is not in 
place yet that would ensure permanent access. 

FRANGAKIS 

• Agrees that human error is far more prevalent than technology error. Key problems in digital 
preservation are infrastructure, technology, and media. Humans need to learn to live, exist, 
and operate in a digital world. It's still very new to us as compared to the print world. 



4. What key policy, organizational, economic, and other non-technical issues need to be addressed or 
solved to facilitate digital preservation? 



SMITH 

• Digital Archiving Task Force Report met with consensus in the community. CLIR has been 
lobbying people to pay attention to these issues. No single community has stepped forward and 
said this is our problem and we are going to work on solving the problem. Therefore, one of 
difficulties is that organizations that collect, preserve, and disseminate information, as opposed 
to create information, find themselves in this digital world in which the preservation of that 
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information must be thought about at the creation stage, not after the fact. Need to forge 
partnerships with the computer science industry, publications industry, scholarly and scientific 
publishing communities to address some of these issues. 

• One of perhaps intractable core infrastructure problems is the issue of creating a failsafe 
archives mechanism for materials that disappear from the web. What happens when 
information is created and the people who created it do not have responsibility for preserving 
it? Who is going to authorize a failsafe archive that is going to take and preserve that 
information for the public good? This would be the equivalent of libraries, but so far, it doesn't 
exist and no one has expressed interest in creating it at the Governmental level. 

• CLIR's role in above: Not in a position to do much more than alert people about the problems. 
NAL and literature that agriculture creates are one of few examples where this failsafe archive 
might work because NAL is a national library dedicated to one type of literature. Not the case 
with other literatures except for medicine (NLM). CLIR is looking for partners like ARL to 
address this issue, but has not made much progress. CLIR has been fairly effective in talking to 
National Science Foundation (NSF) in getting their second round of digital library initiatives 
grants to address the issue of preservation as a distinct issue. No luck with archiving part. 

FRANGAKIS 

Refer to the report Framework for the Preservation of and Permanent Public Access to USDA 

Digital Publications by Paul Uhlir, November 1997 (listed in bibliography). 

• Three areas of issues: 

(1) management structure and organizational relationships within and outside USDA, 

(2) funding of program on a permanent basis (keeping in mind the need to minimize costs of 
access and retrieval to information users), and 

(3) identification of legislative or administrative actions or policies required to implement a 
digital publications preservation program. 

Sub-Issues: Needs and Considerations for USDA 

• Inventory and life cycle management: a comprehensive inventory of all departmental digital 
information products and how they are being managed needs to be conducted. A system for 
tracking the creation of each new USDA digital information product that is intended for public 
distribution needs to be recommended. 

• Technical requirements: identification of acceptable document formats and media, and related 
standards, for long-term retention; development of processes for transferring all digital 
publications from old storage media to new media; establishment of one or more separate back- 
up facilities for all digital publications; review and establishment of system security protocols; 
review and establishment of system interoperability requirements; and identification and 
review of other permanent digital preservation and access initiatives. 

• User access and retrieval: provide equitable access and retrieval services to all potential users; 
minimize technical, regulatory, and cost barriers to access and retrieval; assure the integrity of 




the information that is made publicly available; make the information as easy to find and use as 
possible, with directories and documentation (metadata), consistent with the Government 
Information Locator System, while protecting confidential or proprietary information; and 
establish a means for users to provide feedback and a mechanism for responding to user 
feedback. 

• Status: Moving ahead with implementation. USDA CIO accepted the report, and under her 
guidance NAL established a national steering committee made up of representatives from 
USDA and from agribusiness, research library community, USAIN, Federal partners, etc. 

• Group will meet on a quarterly basis for first 2 years. 

• Will establish test groups to explore issues such as inventory and life cycle management, 
technical requirements, and user access and retrieval, as well as funding issues. 

• Hoping to get funding for a pilot project and then take entire framework and test it on 
an agency within USDA to see how manageable Framework will be for full-scale 
implementation. 



5. What technological strategies or models have various organizations such as the Association of 

Research Libraries, the Digital Library Federation, National Archives, etc., identified to address these 
problems? Evelyn, one of the NCLIS staff mentioned that NAL has established a structure or 
framework that addresses this problem. Could you please talk more about that? If you have any 
handouts you can fax to us, that would also be helpf^ul. Abby, can you describe some of CILR's 
recent efforts to address the issue of digital preservation, including the survey by Jeff Rothenberg of 
the RAND Corporation? 



SMITH 

Three CLIR initiatives: 

• Commissioned report by Jeff Rothenberg from RAND Corporation on emulation. [Emulation is 
the process of imitating one system with another so both accept the same data, execute the same 
programs, and achieve the same results.) Report complete and may be published by January 
1999. Since report is highly controversial, CILR will partner with National Research Council 
to convene a group of computer scientists to engage Rothenberg on issues of emulation to 
stimulate research. Report describes the weaknesses of migration and the strengths of 
emulation and sets up a research agenda to develop emulation. 

• Commissioned an analysis of migrating file formats to do a risk assessment associated with 
those file formats during migration. Study commissioned from Cornell using data from the 
Mann Library (agricultural library) and will use numeric file formats and databases and text 
formats. Report will be finished by September 1999 and will include analysis and a template 
that others can use for doing a risk assessment of migration of those file formats. Purpose: to 
stimulate further research. 
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• Identified a computer scientist at Carnegie-Melion University (CMU), John Ockerbioom, who 
has developed a system of file conversion; type of migration that converts web-based materials 
to different file formats, called TOM (Typed Object Model). 

(www.cs.cmu.edu/afs/cs.cmu.edu/user/spok/www/defense/index.html). He developed this as part of 
his thesis. Working with CMU to see if they can bring his concepts into fuller application to do 
an assessment about its scalability. 

• Log on to publications on CLIR site, which summarize Rothenberg report. Water's report 
addresses definition of digital preservation. 



6. What do we know about specific file formats or mediums that might facilitate digital preservation 
such as SGML, CD-ROM, etc.? 



SMITH 

Nothing to say about this 

FRANGAKIS 

• For conversion efforts from paper to digital images, SGML serves as an important descriptive 
markup tool. Thinks it will be valuable to them. CD-ROM serves specific functions in NAL’s 
Preservation Program but right now has a limited life expectancy. NAL is looking for things 
that are non-proprietary, platform independent, things that will allow user full access to the 
content of digital products. They know that media will continue to change. 



7. Are there any important preservation issues that we have not addressed in the above-listed questions? 
If so, please discuss them. 



No. 



8. Could you please refer us to any important articles on this topic that have been published in the last 
year? 



1. Margaret Hedstrom at the University of Michigan School of Information believes that there is a 
reliable way of preserving, with predictable levels of loss, migration of digital information, 
through ASCII. It happens now. She is a leading authority in the field. 

2. Check ARL's website. 

3. Coalition of Networked Information - Dedicated to computer use in education.www.cni.org 

4. White paper on access authorization. Developing infrastructure for digital libraries. 

5. www.RLG.org/preserv - (includes information onHedstrom's research). 

6. Reference Model for Open Archival Information Systems: 
http;//ssdoo. gsfc.nasa.gov/nost/isoas/ref_model.html 

White Book, issue #4, Sept. 1998- preservation of digital information; technical 
recommendation for use in developing consensus on what's required of any archive to provide 
permanent preservation. Hoping to turn this into an ISO standard, but now just in draft form. 
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Interview Questions for Information Resources Specialist: 

John Bertot 
(November 18, 1998) 



1. What are the primary obstacles to successful information resources management (IRM) practices in 
the Federal Government, in priority order? What changes should occur to eliminate or alleviate the 
barriers? 

2. In your article on the impact of Federal IRM on agency missions, you mention the reinvention of 
IRM to be the key link between agency information and agency performance. Can you describe a 
small and a large Federal agency that currently meet this goal in spite of the lack of a concentrated, 
coordinated Federal IRM policy? Why are they more effective than other agencies? 

3. We have found in our interviews with Federal agency persoimel that many agencies have not come to 
terms with two important issues: information life-cycle management, and the concept of permanent 
public availability of electronic Government information. What are some of the larger policy issues 
that have prevented agencies from addressing these important issues? 

4. In our site visits to Federal depository libraries in the D.C. metropolitan area, we are keenly aware 
that the problems and issues faced by FDLs here are different than they might be for FDLs located in 
more isolated, rural areas. Based on your survey of public libraries connected to the Internet, what 
are some of the key concerns or problems faced by users who want to access electronic Government 
information who live in small, isolated communities with limited resources? 

5. Based on your experience in working with Federal Government agencies that are analyzing their web 
usage, what are the key questions they want to answer and how are they using the data? Are they 
analyzing the websites for technical or content-related purposes? What techniques, other than log 
file analysis, are being used? What agency units or departments (e.g., IT, program areas, CIO) are 
involved in the process? 
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Interview Questions for Information Resources Specialist: 
Charles McClure 
(November 24, 1998) 



1. What is the current status of IRM policy since your 1994 article, "Federal Information Resources 
Management: New Challenges for the Nineties?" Specifically, has 0MB or another appropriate 
agency begun addressing the issue of developing a broad vision that reflects the evolving role of IRM 
within the Government with general guidelines and standards for all Federal agencies to better 
manage the life cycles of information? If so, how? 



2. We have found in our interviews with Federal agency personnel that many agencies have not dealt 
with two important issues: information life-cycle management, and permanent public accessibility of 
electronic Government information. What are some of the larger planning, policy, and organizational 
issues that are preventing agencies from addressing these important problems? 

3. Could you talk a little more about the design-based assessment for evaluating Government websites 
that you described in the 1997 Proceedings of the 60th ASIS annual meeting? What were the 
technical and policy problems in the design-based assessment? What specific policy issues did you 
assess? 



4. What is the status of electronic record management (ERM) guidance for Federal agencies since the 
1998 conference? Is there another conference planned next year? What specific guidance are the 
NARA Working Group and other agencies planning? 



5. What would you say are the top three Federal IRM challenges in the next decade? 
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Summary of Notes 
Interview with 

John Bertot, Associate Professor, SUNY/Albany 



1. What are the primary obstacles to successful IRM practices in the Federal Government, in priority 
order? What changes should occur to eliminate or alleviate the barriers? 



• IDRM is not on the radar for top-level agency managers and it will never be raised up to the 
point of where it matters. 

• IRM has been lost in the transition to the CIO. CIO is the next iteration of IRM. There has 
been 20 years worth of talking and it has never seemed to make it out of the administrative 
trenches of the agencies. Typically, IRM has been a low-level position located within the 
printing, reprographic, or records management units of agencies. IRM is not viewed as 
strategic or long-range function. 



2. In your article on the impact of Federal IRM on agency missions, you mention the reinvention of 
IRM to be the key link between agency information and agency performance. Can you describe a 
small and a large Federal agency that currently meets this goal in spite of the lack of a concentrated, 
coordinated Federal IRM policy? Why are they more effective than other agencies? 



The article is based on Bertot’s dissertation. 

The purpose of the survey was to get an internal assessment on what IRM is trying to do, and to get 
an external assessment of what IRM is doing. Bertot tried to compare the two in relation to 
strategic planning. 

Generally, those agencies that understood IRM tended to be the smaller agencies. There is a scale 
factor, and much more attention was given to IRM in the smaller to mid-sized agencies. FDIC and 
the Peace Corps were doing some interesting things. 

Other factors relating to size: 

• Small to medium-sized agencies have fewer programs and staff; with fewer administrative 
layers, there are fewer communication and organizational barriers. 

• One can more easily work collaboratively in a small organization. 

• Top administrators are not as removed from day-to-day operations and can ideally 
participate more in implementing new initiatives because they have a vested interest in the 
projects’ working. 

e There is less oversight from OMB and Congress for smaller agencies. The smaller agencies 
tend to have less mandated legislation that can interrupt work, so they tend to have higher 
motivation to finish projects. 
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Larger agencies tend to have the greater expertise. Smaller agencies may have better levels 
of management, but they do not always have the experts. 



Models 

• As far as larger agencies were concerned, Treasury was moving along. However, many have 
a central agency component that is not very powerful, althoughsubagencies might be very 
powerful. For example. Treasury has IRS and the FBI, and they are pretty powerful players. 

• Another agency that has done a great deal of work with IRM is EPA. EPA, along with AID, 
are strange models. They have large data shops, but they are all contractors. The model that 
is adopted for information and information technology management makes a difference. 
Whether IRM is in-house or out-sourced has a real impact on how it is implemented inside 
the agency, and the choice of contractor really matters. Another aspect that makes EPA 
unique is that a large portion of their system management function occurs in North Carolina. 



3. We have found in our interviews with Federal agency personnel that many agencies have not come to 
terms with two important issues: information life cycle management, and the concept of permanent 
public availability of electronic Government information. What are some of the larger policy issues 
that have prevented agencies from addressing these important issues? 



The biggest barrier to successful implementation of IRM is that agencies do not view 
information as a resource. There is little or no understanding of the concept of information as 
a life cycle; it's not linear. 

IRM policy initiatives and legislation do not fully address the life cycle of information. It is 
mentioned in some of the policies developed within the last 20 years, but not adequately 
addressed. Most policy initiatives focus on the technology side of the issue, probably because 
it is tangible. 

The web has created problems that have not been handled. Many agencies believe that if it is 
up on the web, it has been published. Along with the pressure over Title 44 Reform, there is 
no discussion of preservation and public accessibility. Should we move to an electronic FDL 
program? What does that mean and how will that work? 

GPO is under attack for being deficient in distributing Government information to the 
public. One reason is because GPO (centralized print environment) is so slow, and the 
technology allows distribution to be handled more efficiently (decentralized, electronic 
environment). Agencies are under the gun to cut costs, so by putting information on their 
websites and contracting printing jobs with outside sources, they don’t have to go through 
GPO. 

Going electronic does offer potential. The back end means, however, that anyone with access 
is a vehicle for getting Government information. Putting information up on the web does cut 
costs, but we have not figured out a systematic way of distributing Government information 
to the public, making sure it is preserved for posterity and provided to the public on a long- 
term basis. 
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4. In our site visits to Federal Depository Libraries in the D.C. metropolitan area, we are keenly aware 
that the problems and issues faced by FDLs here are different than they might be for FDLs located in 
more isolated, rural areas. Based on your survey of public libraries connected to the Internet, what 
are some of the key concerns or problems faced by users who want to access electronic Government 
information who live in small, isolated communities with limited resources? 



Recently Bertot conducted some research in rural Pennsylvania to study public libraries. Public 

libraries in rural communicates face large problems with access and technology. 

• These areas are composed of populations that tend not to have computers in the home, so 
they rely totally on the library for Internet access. 

• The public libraries tend to only have one station in these rural areas. 

• Computers are slow; libraries have 56 K modems, but they do not necessarily have access to a 

56 K Internet provider. 

• Patrons can only reserve computers in half-hour time slots. 

• Libraries may or may not have access to print equipment. 

• Staff training is minimal due to cost and little access to computers. (Models like GPO Access 
have been useful, but patrons and librarians still need one site for access to all Government 
agencies rather than many individual websites. Most agencies have more than one site.) 

• Staff are competing with patrons for access because there are only one or two computers. 

• Libraries cannot always pay for printing services. Some are passing the cost to the user, but 
they are trying to avoid that approach (e.g., first 5 pages are free and then it is 10 cents per 
page). 



5. Based on your experience in working with Federal Government agencies that are analyzing their web 
usage, what are the key questions they want to answer and how are they using the data? Are they 
analyzing the websites for technical or content-related purposes? What techniques, other than log file 
analysis, are being used? What agency units or departments (e.g., IT, program areas, CIO) are 
involved in the process? 



• Bertot has not seen agencies doing very much with their web statistics. Part of the reason is 
that they do not want to make the information public. For example, one agency had a request 
for all of their agency log file records. They panicked and rejected the request on the 
condition of privacy. If one can access the log, one can get IP addresses, and they were afraid 
that someone would use this information as a reverse directory mailer. 

• A second reason is that sometimes it is difficult to get the statistics if a different 
administrative unit within the agency is managing the website. They won't necessarily turn 
them over to the unit that needs the statistics because it crosses administrative barriers. 
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A third reason relates to records management of log files. Should we "schedule" log files for 
NARA? NARA doesn’t want this to happen either. They would then have to schedule the 
information for retention. However, this raises the question of whether the logs are public 
information. 

It was not until recently that there was a demand to look at web statistics as a management 
and strategic decision-making tool. Managers are still learning how to use them. Right now, 
these statistics are used primarily by network and system administrators. 

Bertot wonders how many agencies are doing web analysis and evaluation given Circular 
130-A, which cautions agencies not to do so if it creates a paperwork burden for them. 



Summaty of Notes 
Interview with Charles McClure, 

Distinguished Professor, School of Information Studies, Syracuse University 



1 . What is the current status of IRM policy since your 1 994 article, “Federal Information Resources 
Management: New Challenges for the Nineties”? Specifically, has 0MB or another appropriate 
agency begun addressing the issue of developing a broad vision that reflects the evolving role of IRM 
within the Government with general guidelines and standards for all Federal agencies to better 
manage the life cycles of information? If so, how? 



IRM policy came and went and no one noticed: 

• GSA is mounting its CIO university effort to provide education and training to CIOs. 

• IRM in Government policy is now whole world to CIO. 

• Many agencies do not now know what to do with IRM. 

• ITMRA (Information Technology Management Reform Act of 1996) — McClure thought this 
policy would strengthen IRM, but in reality, it took responsibility away from existing IRM 
people and gave it to CIO; it gave more attention to technology management. 

• A few agencies don't know what to do with IRM staff since CIO is on board; in other 
agencies there is conflict between IRM and CIO functions. 

• Eighty-two percent of technology efforts in agencies are currently focused on Y2K efforts. 



2. We have found in our interviews with Federal agency personnel that many agencies have not dealt 
with two important issues: information life cycle management, and permanent public accessibility of 
electronic Government information. What are some of the larger planning, policy, and organizational 
issues that are preventing agencies from addressing these important problems? 



• There is no staff or time to devote to standards and interoperability. 

• Even if agencies had staff and time, staff need to upgrade skills and knowledge. 

• Information life cycle and permanent public accessibility are not priorities for agencies; they 
don’t seem to understand the issues. 

• For example, GPO Reform Bill is dead in the water; no one in Congress cared about it. 

• No one is concerned about long-term accessibility. 



3. Could you talk a little more about the design-based assessment for evaluating Government websites 
that you described in the 1997 Proceedings of the 60th ASIS annual meeting? What were the 
technical and policy problems in the design-based assessment? What specific policy issues did you 
assess? 



They are now using more advanced website methodology (i.e., a 4-legged approach): 

• User-based: Usability testing; simulates user searching that is videotaped. With fairly 
sophisticated graduate students, they use scripted search analysis with a range of criteria. 
System and design staff are showing videotapes to so they can see the problems with 
searching information on specific sites. Agencies have used different audiences to do testing 
based on objectives and purpose of sites. 

• Log analysis: Using in-house scripts beyond WebTrends and Log Tracker that allow them to 
do cross-file analysis with access vs. error and browser files. Perl scripts allow them to dump 
selected variables in log files into SASS or SPSS. Commercial products do not do cross-log 
analysis well. 

• Policy analysis: Internal policies (who's in charge) and external policies (e.g., Freedom of 
Information Act, public access, privacy issues, etc.). 

• Management assessment: How is agency department organized for web maintenance and 
evaluation? Information is gathered through interviews and focus groups with managers. 



4. What is the status of electronic records management (ERM) guidance for Federal agencies since the 
1 998 conference? Is there another conference planned next year? What specific guidance are the 
NARA Working Group and other agencies planning? 



Update: McClure and Tim Spreche are working on a new project, PARS- Public Access 
Rating System. The purpose of this project is to create a core set of performance measures 
and indicators (now being developed for four Government agencies) with public access 
criteria to help agencies rate their websites. Agencies will be able to determine the degree to 
which the site is publicly accessible. It's difficult to sell ERM by itself, so they are taking a 
public access approach. 

National Archives hasn’t done much. Court ruling delayed 6-8 months. 



5. What would you say are the top three Federal IRM challenges in the next decade? 



IT management policy development is on hold due to Y2K. No one currently knows how 
good or bad preparation is for this. 

How best to integrate and coordinate IT and IT management. Agencies do not have a good 
handle on this yet. 
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Issues of interoperability and standards that cut across all agencies. Need to be able to access 
Government information horizontally rather than vertically. (For example, for public access 
sites; GILS, gov.doc I, and one more; no way public can access specific information from one 
point of entry. GILS does not work the way it was originally conceived.) 

Lack of money for training and education. IRM graduate students' degrees are useful for 
about 1-2 years. After that, their skills are 50 percent out of date. Government agencies have 
well meaning people who don't have the knowledge and skills to implement policies. For 
example, agencies say they don't need to send copies of all products to GPO because they are 
on their website. Then you ask them will it be there in 6 months and they have given no 
thought to this issue. 



Appendix I 

Sample Agency Meeting Agenda 
Electronic Government Information Products Assessment 
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Sample Agency Meeting Agenda 
Electronic Government Information Products Assessment 



Meeting with Health and Human Services Agency 
Representatives and Product Respondents 

Tuesday, September 15, 1998 
9:15 a.m. 



9:15 


Introductions 


Westat, NCLIS, and HHS Personnel 


9:25 


Background and Purpose of the Meeting 


Westat and NCLIS 


9:30 


General Questions and Answers 
• Sections A and B on the Questionnaire 


Westat, NCLIS, HHS Personnel 


10:00 


Q and A to Sections C and D 
• Review Data Collection Procedures 


NCLIS, Westat, HHS Personnel 


10:30 


Break 




10:45 


Agency Discussion Questions 

(Discussion questions posted on NCLIS web site) 


Westat and HHS Personnel 


11:45 


Adjournment 
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Appendix J 
Task 16 

Assessment of Electronic Government Information Products 



Statement of Work 



Section J: Background and Objectives 



Background 

An "Assessment of Electronic Government Information Products" (hereinafter referred to as the 
Assessment ) is a research study being conducted through an interagency agreement between 
the National Center for Education Statistics (NCES) and the U. S. National Commission on 
Libraries and Information Science (NCLIS) on behalf of the Superintendent of Documents, U.S. 
Government Printing Office (GPO), pursuant to an Interagency Agreement between NCLIS and 
GPO, approved by the Joint Committee on Printing (JCP). 

Information gathered from this assessment is to be used by the Superintendent of Documents to 
facilitate improved public access to Federal Government information made available to Federal 
depository libraries and the general public through the Federal Depository Library Program 
(FDLP). The assessment will 

(1) identify medium and format standards that are the most appropriate for permanent public 
access; 

(2) assess the cost-effectiveness and usefulness of various alternative medium and format 
standards; and 

(3) identify public and private medium and format standards that are, or could be used for 
products throughout their entire information life cycle, not just at the dissemination or 
permanent public access phase. 



Objectives 

The contractor shall undertake research and data collection from Federal agencies in all three 
Branches, as well as solicit the opinions of selected knowledgeable experts. The contractor shall 
also complete an analysis of the data and opinions for the purpose of interpreting their general 
meaning and significance, including identifying broad emerging trends and patterns, and 
documenting findings, conclusions and recommendations in a deliverable final report. 

More specifically, for a cross-section of Government information products, the goals are to: (1) 
determine in which format(s) and medium(s) such products are now produced, using which 
standard(s) if any (whether promulgated by official standards-setting bodies such as ISO, NISO, 
ANSI or FIPS, or voluntarily adopted through common agency practice, such as the use of file 
formats such as PDF or TIFF , or Microsoft Word or dBase); and (2) assess agency future plans 
for new or changed products, including the medium(s) and format(s) in which they will be 
disseminated for permanent public access. 

The Superintendent of Documents will use the results of this work effort to continue to plan and 
implement the transition to a more electronic FDLP. The 5 major specific objectives are: 



> First, with respect to electronic publishing practices and plans of Federal agencies (including 
ways in which the FDLP can best accommodate them), the objective is to provide an analysis 
of current practices as well as future plans for creating, disseminating, and providing 
permanent public accessibility to electronic information products, and to identify the 
standards for software, and electronic mediums and formats that are or will be used 
throughout the product's information life cycle, from creation to archiving, but especially by 
the time of the permanent public accessibility phase; 

> Second, with respect to cost effectiveness of various dissemination mediums and formats that 
are, or could be, utilized, the objective is to gather information on standards (whether 
mandated or consensual) that will assist the FDLP in making near-term decisions regarding 
the cost-effectiveness of alternative mediums and formats for all FDLP participants. This 
information should also assist participants in long-term planning for permanent public 
accessibility, and the collection and analysis of overall information life-cycle costs; 

> Third, with respect to the practical utility of various electronic mediums and formats to 
depository libraries and the public, the objective is to identify preferred standards used in 
various mediums and formats that depository libraries will need to support; 

> Fourth, with respect to utilizing standards employed in mediums and formats that can be used 
throughout all stages of the information life cycle (including creation, composition, computer 
terminal display, encryption, secure digital signature with non-repudiation and secure 

transmission capabilities), electronic dissemination, but especially permanent public 

accessibility , the objective is to assess standards for basic security services in order to 
provide for secure and reliable transmission and document interchange; and 

> Fifth, with respect to standards that are being developed and used in the private sector , the 
objective is to identify existing and planned standards for the purpose of determining what 
the FDLP must do to accommodate their adoption by the agency in terms of 
hardware/software requirements, staff and user education and training, and budgetary 
impacts. 

For the purposes of this survey an agency Web site is considered an electronic information 

service , and one or more products may reside on the service. Web sites, per se, are not 

considered individual Government information products. 
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Section II: Subtasks 



The contractor will be provided background materials to assist in conducting the required work, 
including a suggested data collection instrument, examples of completed forms, selection criteria 
for finalizing the list of agencies and products to be surveyed, and additional materials (see 
Appendix C). Working with representatives of NCLIS and GPO, the contractor shall undertake 
the following specific activities and complete them by the scheduled dates shown below. A 
detailed explanation of each activity appears in Appendix C: 



Activity No. 


Activity Title 


1 


Prepare a Plan of Action 


2 


Develop a Data Collection Plan identifying preliminary agency and product 
selections, and a schedule of interviews, focus group meetings and site visits, 
using Appendix B as a guide 


3 


Conduct interviews, focus group meetings and site visits; revise preliminary 
agency and product selections if necessary 


4 


Pre-test the Data Collection Instrument (Appendix A); make changes if 
necessary 


5 


Create a Baseline Inventory Products Data Base ready to populate with actual 
data 


6 


Collect the data from Federal agencies, experts, and literature references 


7 


Prepare tabulations and summaries based on the populated data base and 
analysis 


8 


Kecoiiect and retabulate if/as necessary 


9 


Prepare a draft Report documenting findings, conclusions and 
recommendations, and provide oral briefings on request; obtain feedback from 
Government 


10 


Prepare a final Renort and submit to the Government 



O 

ERIC 


184 

J-5 



Section III. Deliverables And Key Events Schedule 



Deliverables, in terms of their delivery in weeks after award, are shown below. For each of the 
deliverables, the contractor shall provide six copies to the COTR. [This is the original schedule 
and does not reflect the changes that were later negotiated between NCLIS and Westat.] 



Deliverable 


Weeks After Award 


Government-contractor planning meeting 


1 


Plan of Action approved 


2 


Anoroval of prelim, agency/contact list 


3 


-T ± 1 K 

Completion of inter views/meetings/site visits 


12 


Approval of Data Collection Plan 


12 


Aeencv data collection instruments transmitted to agencies 


13 


Actual data collected 


16 


First oral briefing 


17 


Tabulations/summaries completed 


17 


Recollection and retabulation completed 


19 


Second oral briefing 


19 


Draft Report submitted 


20 


Final oral briefing 


22 


Final Report submitted 


25 
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Section IV - Management Reporting Requirements 

The contractor shall provide the following reports for the purpose of maintaining a detailed 
record of work ordered and funds spent on this task order: 

Cost Proposal; 

Schedule of Deliverables 

Task Order "Log" or menu [monthly]; 

Monthly Cost Report; 

Task Invoices; and 
Reimbursement Report [monthly]. 

Details pertaining to these and ancillary reporting activities follow. 

Cost Proposals 



The contractor shall provide a cost proposal to accompany the contractor's response to the work 
identified. The cost proposal will be reviewed and approved by the COTR and the contracting 
officer, who will communicate the approval to the contractor. [Should there be a need to modify 
the cost proposal, the COTR and the contracting officer will communicate that instruction to the 
contractor's Project Director, who shall submit a revised cost proposal for approval.] 

Task Order **Log" 

The contactor shall provide a monthly report indicating the tasks ordered by the COTR and the 
contracting officer to date and showing the internal (to the contractor) tracking number assigned 
to each. The contractor shall insure that this report includes such summary information as: the 
contractor staff member assigned as task leader; date(s) associated with the task, including 1) 
date assigned to the contractor; 2) date scheduled for completion of the task; and 3) upon 
completion, date of invoice(s) for services rendered in support of the task. 

Monthly Cost Report 

The contractor shall provide a monthly report to the cotr and the contracting officer as specified 
under the terms and conditions of the contract and detailed in the proposal. As part of the 
monthly report, the contractor shall include a list of deliverables and their anticipated due dates. 

Reimbursement Report 

The contractor shall provide the COTR and the contracting officer with a status report on 
reimbursement activities. The contractor shall insure that the report includes the names of the 
consultants and participants and any others who are to be reimbursed for expenses incurred, as 
applicable and approved by the COTR and the contracting officer. The contractor shall include: 
1) the subtask for which the person is subject for reimbursement; 2) the amount subject to 
reimbursement for each subtask in which the person is a participant; 3) the date each participant 
submitted the necessary reimbursement form for each subtask; 4) the date the participant is 
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scheduled to be issued reimbursement for each subtask; and 5) cumulative reimbursement totals 
to date, updated. 



Period of Performance 

The period of performance for this task is eight months from the date the award is signed. 
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